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ABSTRACT 



This research examines the evaluation performed by the Navy International Programs 
Office on United States Navy programs proposed as candidates for international 
codevelopment agreements. Primary research consists of a spreadsheet analysis of 22 
program evaluations conducted by the International Programs Office during 1990-1991. 
Secondary research recounts the history of the U. S. Government's international armaments 
cooperative efforts from World War 11 to the present, explores current issues, and examines 
Department of Defense and Department of the Navy policy and procedures governing 
international cooperative programs. 
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I. 



INTRODUCTION 



A. BACKGROUND 

On October 5, 1990, the Navy International Programs Office 
(IPO) instituted a five-step evaluation method using a table 
of 14 success criteria, grouped in three sections, to evaluate 
Research and Development (R&D) programs proposed as candidates 
for international codevelopment agreements between the U. S. 
Government and allied nations. Before the development of the 
success criteria, candidate programs were qualitatively 
evaluated using a lengthy worksheet. With the introduction of 
the success criteria, candidate programs were evaluated to 
identify those programs with a low likelihood of success in 
the international arena, and so direct the IPO's resources at 
those candidate programs more likely to succeed. 

The evaluation consisted of applying the 14 success 
criteria twice to each candidate program, once to rate U. S. 
participation in the program and once to rate allied 
participation. Six strength ratings were then calculated, one 
for each section of criteria for both the U. S. Government's 
and the foreign participation. These paired ratings were 
compared to each other; based on their "degree of 
convergence," a weighted factor ranging from 0.5 to 1.5 was 
then applied to each paired ratings. The three resultant 
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weighted rates were added to arrive at a final composite 
strength rating for each candidate program. Eighteen 
candidate programs were thus evaluated in the following 
months; they were ranked in descending order according to 
their composite strength ratings. 

On July 30, 1991, IPO revised the success criteria, 
dropping two items, substituting in for two others, and 
regrouping the criteria into four sections vice three. 
Generally, the five- step evaluation process remained 
unchanged. The new procedure did call, however, for only a 
single "strength matching" weight to be applied to the entire 
evaluation to determine the candidate's composite strength, 
this instead of the area "degree of convergence" weights. 
Four additional candidate programs and six of the original 18 
candidate programs were evaluated using the revised criteria 
and procedures. The results revealed some anomalies: not 
only did the four additional candidate programs score 
generally lower than the six original programs, but the 
priority ranking of some original programs was reversed. 

B. OBJECTIVES 

The general objective of this research was to examine the 
impact that the revised evaluation criteria and procedures had 
on the priority ranking of candidate programs, and so verify 
the accuracy of the revised criteria and procedures at 
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indicating a program's potential for success as an 
international codevelopment agreement. 



C. RESEARCH QUESTION 

The primary research question addressed by this research 
is as follows: 

• How can the evaluation conducted by the Navy International 
Programs Office on programs being considered for 
nomination as international codevelopment agreements be 
refined so as to offer a more accurate "front end" 
analysis of the candidate programs? 



Secondary research questions addressed by this research in 
support of the primary research question are as follows: 

• What has been the U. S. Government's participation in 
international armaments agreements since World War II? 

• What are the current issues affecting efforts at 
international armaments agreements? 

• What are Department of Defense and Department of the Navy- 
policies and procedures regarding international 
codevelopment agreements? 



D. SCOPE, LIMITATIONS AND ASSUMPTIONS 
1. Scope 

This research examines the evaluations performed by IPO on 
22 candidate programs from October, 1990 through July, 1991. 
Data for this research consist of the original and revised 
success criteria, the 18 evaluations conducted using the 
original criteria and procedures, and the ten evaluations 
conducted using the revised criteria and procedures. 
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Limitations 



2 . 

The only significant limitation affecting this research 
was an inability to access the computer program that generated 
the evaluation sheets for the International Programs Office; 
access to the computer was denied for security reasons. In 
lieu of access, computer printouts representing all the 
evaluations performed by the IPO during the period of the 
research were provided. 

3 . Assumptions 

The research assumes that the reader has a basic 
understanding of spreadsheet calculations and construction. 

E. LITERATURE REVIEW AND METHODOLOGY 

A literature review was conducted to determine the history 
of the U. S. Government's participation in international 
armaments agreements since World War II, and to determine 
current issues affecting international codevelopment efforts. 
A review of relevant Department of Defense instructions and 
Department of the Navy reports was conducted to determine 
official policy and procedures. 

The methodology for this research consisted primarily of 
a personal interview with Mr. Frank D. Kenlon, Director for 
International Programs with the Navy International Programs 
Office in September, 1991. Extensive use is made of a 
spreadsheet analysis of the individual evaluation sheets. 
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F . OVERVIEW 

Chapter II, Background, recounts the U. S. Government's 
involvement in international armaments cooperative agreements 
with its allies from World War II to present. It additionally 
explores current issues in codevelopment and coproduction 
agreements, and examines current and proposed Department of 
Defense (DoD) and Department of the Navy (DoN) policy 
regarding international agreements. 

Chapter III, Presentation of Data, displays the data used 
in conducting this research. Tables exhibiting the original 
and revised success criteria, and individual evaluation sheets 
for all program evaluations conducted by the IPO are included. 

Chapter IV, Analysis of Data, reproduces the cell -by- cell 
spreadsheet analysis of the various evaluation sheets, and 
correlates the IPO's success criteria to stated Office of the 
Secretary of Defense (OSD) international program success 
indicators . 

Chapter V offers the conclusions and recommendations 
developed as a result of the research. 
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II. 



BACKGROUND 



A. U. S. PARTICIPATION IN MODERN MILITARY ALLIANCES 

The United States' involvement in its present day military 
alliances is largely a result of the economic and military 
uncertainties that characterized the world in the years 
following World War II. The war levied a terrible toll across 
the Europe and Asia, so that 

. . . the global balance of power after the war would be 
totally different from that preceding it. Former Great 
Powers- - France , Italy- -were already eclipsed. The German 
bid for mastery in Europe was collapsing, as was Japan's 
bid in the Far East and Pacific. Britain . . . was 
fading .... Only the United States and the USSR 
counted, so it seemed; and of the two, the American 
"superpower" was vastly superior (Kennedy, 357) . 

Much of the United States' power and prestige lay in its 
economic might : 

The country's GNP measured in constant 1939 dollars rose 
from $88.6 billion (1939) to $135 billion 
(1945) .... industrial expansion in the United States 

rose at a faster pace . . . than at any period before or 

since .... Among the Great Powers, the United States 
was the only country which became richer- -in fact, much 
richer- -than poorer because of the war (Kennedy, 357-358) . 

By comparison, the former Great Powers (Germany, Japan, Italy, 

France and Great Britain) all suffered greivous economic 

reversals as a result of the war. Germany was divided up into 

four military occupation zones, and later, into two countries; 

its industrial base, devastated by bombing, was dismantled; 

its national income and output dropped over two- thirds between 
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1938 and 1946 (Kennedy, 365) . Japan was similarly occupied; 
its national income fell 43 percent from 1936 to 1946 
(Kennedy, 365). Italy's GNP in 1945 was only 60 percent of 
its 1938 level (Kennedy, 365-366) . France had been plundered 
by the Germans for four years and then suffered from the heavy 
fighting during 1944; its 1945 income was only 50 percent of 
its prewar level (Kennedy, 366) . Great Britain fared better 
than those countries, but, even so, was suffering economically 
at the end of the war; it had depleted its gold and dollar 
reserves, worn out its domestic industrial capability, and was 
heavily dependent on the United States for much of its arms, 
shipping, food and other supplies (Kennedy, 367) . 

The United States' economic might was projected worldwide 
by its military power. At the end of the war, the U. S. 
military was comprised of some 12.5 million personnel (7.5 
million deployed overseas) ; its modern weaponry included 1200 
warships, 3000 bombers, and a monopoly of atomic weapons 
(Kennedy, 358) . Of the European powers, only Great Britain 
possessed a credible military force at the end of the war, 
with forces deployed to 

. North Africa, Italy, Germany, (and) Southeast 
Asia. Despite heavy losses, the Royal Navy possessed over 
1,000 warships, nearly 3,000 minor war vessels, and nearly 
5,500 landing craft. RAF Bomber Command was the second- 
largest strategic air force (by far) in the world 
(Kennedy, 367) . 

If World War II heralded the emergence of the United 
States as a superpower, the same must also be said of the 



7 



Soviet Union. The increasing Soviet influence can be seen by 
its territorial growth during and after the war. Its European 
boundaries expanded 

. . . in the north at the expense of Finland, in the 

center at the expense of Poland; and in the south, 
recovering Bessarabia, at the expense of Rumania. The 
Baltic states of Estonia, Latvia, and Lithuania were 
reincorporated into Russia. Part of East Prussia was 
taken, and a slice of eastern Czechoslovakia . . . was 

also thoughtfully added, so that there was direct access 
to Hungary. To the west and southwest of this enhanced 
Russia lay a new cordon sanitaire of satellite states, 
Poland, East Germany, Czechoslovakia, Hungary, Rumania, 
Bulgaria, Yugoslavia and Albania (Kennedy, 361) . 

In the Far East, the Soviets occupied Manchuria, North 
Korea and Sakhalin and linked up with the Chinese Communists 
(Kennedy, 367) . After the war, Stalin would reduce the size 
of the Soviet Army by two- thirds 

to the still very substantial total of 175 
divisions, supported by 25,000 front-line tanks and 19,000 
aircraft. It would still remain . . . the largest defense 
establishment in the world- -a fact justified ... by its 
need to deter future aggressors and ... to keep control 
of its newly acquired satellites in Europe as well as its 
conquests in the Far East (Kennedy, 363). 

The Soviet military strength belied the country's economy, 

which was shattered by the war. Estimates put the number of 

Soviet deaths as a result of the war at 20-25 million; in 

addition, livestock, farm equipment, the transportation 

infrastructure, and housing were likewise decimated by the 

occupying German Army (Kennedy, 362) . In short, "the Russia 

of 1945 was a military giant and, at the same time, 

economically poor, deprived and unbalanced" (Kennedy, 362) . 
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The ideological splitting of Europe into the Capitalist 
Western bloc and the Communist Eastern bloc became apparent 
soon after the end of the War. It was noted in 1946 and early 
in 1947 that 

. . . various plebiscites and regional elections in the 
German zones (of occupation) were showing "the political 
complexion of West Germany . . . beginning to differ 
markedly from that of East Germany"; the steady 
elimination of any non-Communist elements in Poland, 
Bulgaria, and Rumania was mirrored by the internal 
political crisis in France in April, 1947, when the 
Communists were forced to resign from the government. A 
month after, the same happened in Italy. In Yugoslavia, 
Tito's political domination (in place of the Allied 
wartime agreements about shared power) was interpreted by 
the West as a further step in Moscow's planned advance 
(Kennedy, 375) . 

A two-pronged strategy was developed by the West to 
contain what was perceived as the Soviet threat to Western and 
Southern Europe. The first element of this strategy 

. . . was to indicate to Moscow those regions of the 
globe which the United States "cannot permit ... to fall 
into hands hostile to us." Such states would ... be 
given military support to build up their powers of 
resistance; and a Soviet attack on them would be regarded 
virtually as a casus belli .... The (second and) most 
crucial component of any long-term containment policy 
would be a massive program of U. S. economic aid, to 
permit the rebuilding of the shattered industries, farms 
and cities of Europe and Japan; for that would not only 
make the latter far less likely to be tempted by Communist 
doctrines of class struggle and revolution, it would also 
help to readjust the power balances in America's favor 
(Kennedy, 376) . 

In Europe, the economic aid element of the containment 
strategy was implemented by the Marshall Plan in 1947; the 
military support element would come two years later when, 
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following the Berlin crisis of 1948-49, the North Atlantic 
Treaty Organization (NATO) was created. 



B. U. S. ARMS COOPERATION EFFORTS SINCE WORLD WAR II 

In the first decade following the signing of the NATO 
treaty, the member nations, confronted by the Sputnik launch 
with an eclipsing of Western missile technology, began to 
cooperate on armaments (Taft, 15). An early expression of 
this cooperative trend was 

. the NATO Co-production Program agreed by the 
Alliance in 1957 .... Since then there have been 

literally hundreds of bilateral and multilateral European 
and transatlantic cooperative programs- -co- development, 
co-production, licensed production projects, MOU's, dual 
production, industry- to- industry efforts , outright weapons 
sales, one allied nation to another (Taft, 15). 

Three phases of the Government's arms cooperation efforts 
since World War II can be identified. The first, lasting from 
the end of the war until the mid-sixties, 

. was characterized by grant aid known as the 
Military Assistance Program (MAP) .... the emphasis was 
on the no-cost (to the recipient) transfer of equipment 
directly from U. S. Force's inventories- - initially surplus 
stocks developed as a result of modernization of our own 
forces- -or from additional new production of systems being 
produced for U. S. Forces (Cole, F-3) . 

Funding for the MAP came entirely from the U. S. Congress; the 

funds were administered by a central office, the Director of 

Military Assistance, that did not concern itself with the 

implementation of the programs by the various acquisition and 

logistics Commands involved (Cole, F-3) . 
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These arrangements worked well. Service hardware 
Program Managers had few if any complaints. The equipment 
being furnished was standard U. S. The requirements were 
easily folded into contracts for equipping U. S. forces; 
the funding was U. S. budget authority; and there were no 
issues of R&D recoupment, administrative surcharges, asset 
use charges, agent fees, co-production, offsets, MOUs, 
etc., that became commonplace for providing U. S. arms to 
friends and allies (Cole, F-3) . 

The second phase of arms cooperation overlaps the first 
but can be characterized by 

... a gradual shift from MAP to sales on a government - 
to- government basis usually referred to as Foreign 
Military Sales (FMS) , and to direct commercial sales 
(Cole, F-3) . 

The Foreign Military Sales program 

. . . initially focused on the stronger economies of 
European NATO . . . with the objective of having allies 
obviously able to pay their own way assume larger shares 
of the common defense burden. 

MAP procedures continued for most other countries 
friendly or allied to the United States and also for some 
European NATO allies (Cole, 13-14) . 

Responsibility for FMS was transferred from the Director 
of Military Assistance to the newly established position of 
Director of International Logistics, and later, along with 
responsibility for MAP, to the Director of the Defense 
Security Assistance Agency (DSAA) upon that agency's 
establishment in 1971 (Cole, F-3 - F-4) . Congress began to 
curtail MAP funds, with the result that DSAA attention became 
increasingly focused on FMS transactions; FMS credits became 
the means by which countries progressed from MAP to cash sales 
(Cole, F-4). DSAA (conceived as simply the fund manager and 
resident authority for international arms programs) , 
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eventually became "firmly astride and enmeshed in the 
acquisition and logistics function" (Cole, F-4) . In response, 
the 

. . . OSD acquisition and logistics staffs, the Military 
Departments and Logistics Commands demanded and received 
a greater voice in international arms matters. By 1976 
program direction had become diffused and controversies 
were common (Cole, F-4) . 

A DoD task force, charged by Secretary of Defense Rumsfeld 
with reviewing Security Assistance, recommended on January 14, 
1977 that DSAA should report directly to the Defense 
Acquisition Executive vice the Assistant Secretary of Defense- 
International Security Affairs, but this recommendation was 
never acted upon by the incoming Carter Administration (Cole, 
F-4) . 

The third (and current) phase of the Government's arms 
cooperation efforts is characterized by declining U. S. arms 
exports and an increase in the number of codevelopment 
agreements (Cole, F-4) . An examination of this phase occurs 
in later in this chapter as part of the discussion of 
Rationalization, Standardization and Interoperability (RSI) as 
it applies to NATO. 

C. ADVANTAGES AND DISADVANTAGES OF ARMS COOPERATION 

The most obvious advantage to two or more countries 
entering an arms cooperation agreement is economic. Thomas A. 
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Callaghan notes: 



For centuries commercial economics and trade have been 
moving towards ever-larger markets, providing economies of 
scale and ever more affordable prices .... Defense 
economics has bucked this trend. Here we have ever-larger 
product lines, ever-smaller production runs, small 
national markets, and (consequently) ever-higher prices 
(Callaghan, 61) . 

It follows, then, that an increase in the market for a 
weapon system through arms cooperation agreements would lead 
to fewer product lines, larger production runs, and lower 
prices. This simple fact was 

. . . first appreciated within Europe, and more so than 
in the United States. There were good reasons for this, 
not least of which were the budgetary constraints with 
which the European states were faced, the range of 
conventional operational requirements they had to meet 
with considerable fewer resources, and the comparatively 
small production run each could afford on any one program 
(Edmonds, 11) . 

But this logic holds true for any arms cooperation 
agreement in that all the participating countries would 
experience cost reductions through 

improvements in production arising from the 
learning process, through quantity purchase of materials 
and parts, better use of fixed capital and steady cash 
flows (Edmonds, 11) . 

Two political arguments can be made as well in favor of 
arms cooperation agreements. First, a weapon system that can 
be demonstrably shown to have reduced unit costs would be more 
likely to be accepted by the citizens of a country, especially 
when the risks associated with developing and producing the 
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weapon system could be shared with other nations (Edmonds, 11- 
12). Secondly, a country that participates in arms 
cooperation agreements with its allies 

demonstrates a willingness to adjust purely 
national priorities in the interest of alliance cohesion 
and, arguably, a willingness to further the wider 
political objectives of creating interdependencies among 
. . . allies and . . . establishing an integrated defense 
industry (Edmonds, 12) . 

A final advantage to arms cooperation agreements is that 
it is militarily more desirable for alliance forces operating 
in conjunction with each other to be equipped with 
standardized weapons, logistics and communications systems 
( Edmonds , 14 ) . 

Set against these reasons to enter arms cooperation 
agreements are the pressures inherent in a sovereign state to 
maintain a unique domestic arms production capability; it 
would be disadvantageous for a nation to enter such an 
agreement when it is that government's desire 

to limit reliance upon external sources of 
military equipment, to reduce the burden of imports on 
their balance of trade, and to ensure maximum employment 
for workers in defense and defense- related industries 
(Edmonds, 1) . 

As Shaffer notes, 

cooperation in armaments is not an area of 
national policy isolated from security policy or from more 
general domestic and foreign policy concerns. As a result 
of this linkage, nations desire armament self-sufficiency, 
to the extent economically and politically feasible, in 
order to avoid dependence on others ... in formulating 
and implementing policy objectives (Shaffer, 27) . 
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In addition, compelling reasons of national security could 
often deter one country from entering an arms cooperation 
agreement due to the possibility of sensitive technology 
transfer. 

D. FORCES CONTRIBUTING TO ARMS DIVERSITY 

The earliest attempts at arms cooperation in NATO led to 
a certain homogeneity of weapons among the alliance members in 
that "virtually all military equipment was supplied by the 
United States" (Shaffer, 29). This uniformity, driven by 
necessity as much as the booming postwar American economy, 
soon fragmented. 

Five specific factors have been identified as contributing 
to the push toward NATO weapons diversity. The first of 
these factors is NATO's "overreliance on nuclear weapons to 
the detriment of conventional forces" (Cannizzo, 57) , 
especially in the early years of the NATO alliance: 

Presuming themselves safe from conventional attack by 
the USSR and its allies, (NATO) general purpose forces 
were by and large starved of resources and left to 
deteriorate; the member states were left free to a large 
degree to go their separate, sovereign ways in pursuit of 
national security .... The United States strongly 
advocated this reliance on massive retaliation and did not 
particularly mind the burgeoning proliferation of 
different types of conventional weapons systems (Cannizzo, 
57) . 

The United States' support of a strategic defense of NATO 
should be viewed in light of its then vastly superior nuclear 
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arsenal and its later commitment of conventional forces to the 



conflicts in Korea and Vietnam. 

The second factor that advanced weapons diversity in NATO 
is closely related to the first: 

The United States encouraged the postwar rebuilding of 
European defense industries in order not to overtax the 
"arsenal of democracy" and to help spur reconstruction 
efforts (Cannizzo, 57) . 

A third factor derives from the economic problems 
afflicting the United States beginning in the early sixties 
brought on by an unfavorable balance of trade and the flow of 
gold passing through U. S. borders (Cannizzo, 57): 

The United States patently became more interested in 
forcing the Europeans to buy U. S. equipment and in 
finding ways to offset the cost of stationing U. S. troops 
abroad than it was in fostering weapons cooperation. Such 
policies constituted a source of irritation for the 
European Allies, which was compounded by the "buy 
American" acts .... Not only the United States has 
continuing economic difficulties, but the Europeans have 
as well, which today have helped create a situation more 
compatible with protectionism than international 
cooperation (Cannizzo, 57) . 

Cannizzo identifies the United States' involvement in 
Vietnam as the fourth factor lending to NATO weapons 
diversity: 

The diversion of U. S. interests and attention to 
Southeast Asia clearly reduced U. S. participation in NATO 
affairs and procurement policies . . . the United States 
drew on its stocks of equipment in Europe and . . . this 
left large gaps which remain unfilled even today 
(Cannizzo, 57) . 

Finally, Cannizzo points to the NATO experience with arms 
cooperation projects themselves as leading to diversity. 
Citing the consortium created to manage the coproduction of 
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the F-16 as an example, Cannizzo points to a myriad of 
problems : 



. . . charges of bribery and even hints of murder during 
the bargaining and selection phases; continual cost 
increases; acrimonious debate over unfulfilled promises of 
jobs, income, and other benefits; disagreements over 
third- country sales; and even disgruntlement over the 
plane's performance (Cannizzo, 57). 

Another example of an unsuccessful NATO joint program was 
the attempt by the United States and West Germany to develop 
the MBT-70 battle tank which "ended in failure after more than 
$400 million had been invested" (Kitfield, "Finally, " 77) . 

The current NATO policy of "Rationalization, 
Standardization, and Interoperability" (RSI) is the latest 
expression of the alliance's efforts to promote arms 
cooperation with the hope of realizing 

not only the militarily desirable goal of 
standardizing equipment and harmonizing logistical 
procedures, but also the politically desirable goal of 
alliance solidarity (Feldman, 48) . 



E. THE EVOLUTION OF RSI 

RSI is intrinsically tied to the theory of "structural 
disarmament" as formulated by Callaghan. Callaghan states 
that the Western nations are 

. . . experiencing (a) disarmament phenomenon of more 
and more money producing fewer and fewer weapons, less 
"readiness," and even less combat sustainability. The 
cause is structural . No nation- -not even the US- -provides 
a large enough defense market to develop and produce 
everything itself (Callaghan, 61) . 
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This section will examine the concept of RSI, and will trace 
its implementation through laws enacted by the United States 
Congress and policy directed by the President and the 
Department of Defense. 

A definition of terms is in order: 

"Rationalization" is defined as any action which 
increases the effectiveness of NATO by improving the 
command- support structure and consolidating the various 
national logistical systems. It includes standardization, 
specialization, interoperability, cross-servicing, and 
general cooperation of military and non-military matters. 

"Standardization" refers to the achievement of the 
closest practical cooperation among forces, sharing of 
systems designs, agreement on operational, administrative, 
and logistic procedures, and common tactical doctrine and 
organization. 

"Interoperability" consists of making different weapons 
systems work together by using common or compatible 
components, fuel, and ammunition (Feldman, 50). 

As was previously mentioned, a high level of 
standardization was achieved in the early years of NATO by the 
simple fact that most weapons were provided by the United 
States. Continued economic aid to the European allies by the 
United States led to a rebuilding of European industry so 
that, by the late 1950s, "European industry was sufficiently 
recovered to permit the independent development of exclusively 
European weapons systems" (Feldman, 51) . 

The Warsaw Pact modernization of the 1970s and the 
realization that NATO was wasting up to $15 billion a year 
because of weapons diversity led to a renewed call for 
standardization in conjunction with the rearming of NATO 
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(Feldman, 51) . But purely national concerns often overrode 
alliance concerns so that 

. . . NATO leaders found it increasingly difficult to 
convince parliaments to spend large amounts of foreign 
exchange on the purchase of weapons from abroad, because 
of their fear of generating domestic unemployment. 
Therefore, most NATO members began in the 1970s to demand 
economic compensation in the form of "offsets" such as 
coproduction or countertrade as a condition of purchase 
(Feldman, 51) . 

In the United States, the Government began to take the 
initiative in promoting RSI. An amendment to the Defense 
Appropriation Authorization Act of 1975 called for the 
Secretary of Defense to ascertain the cost of the failure to 
standardize weapons in NATO and to develop a plan for arms 
cooperation (Feldman, 51) . The 19 76 Nunn amendment encouraged 
reciprocal procurement of equipment between the U. S. and 
other NATO nations; authorized the SECDEF to waive the "Buy 
American" law in the interest of NATO standardization and 
interoperability; and enabled the establishment of bilateral 
reciprocal procurement MOUs between the U. S. and the European 
nations of NATO (Naval, 25). 

President Ford introduced the two-way street concept to 
the Europeans at the 1976 NATO summit, and advised them that, 
to compete effectively with U. S. defense industry, the 
Europeans would have to sacrifice some national concerns and 
combine their resources (Feldman, 52) . 
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The Carter administration approached arms cooperation from 
two almost diametrically opposing views. On the one hand, the 
administration 

. . wanted to greatly decrease the U. S. sales of 
weapons around the world, thus avoiding a reputation as 
"Merchants of Death." Correspondence, dubbed the "Leprosy 
Letter," was sent to all U. S. embassies early in the 
Carter administration prohibiting support and assistance 
to U. S. defense industry corporations trying to sell 
overseas. Legislation was enacted by Congress disallowing 
foreign marketing expenses to be charged as a valid cost 
of doing business (Naval, 23). 

On the other hand, the Carter administration encouraged arms 
cooperative agreements with allies; in particular, Under 
Secretary of Defense for Research and Engineering William 
Perry greatly furthered the cause of RSI. Through Perry's DoD 
Directive 2010.6, a three-pronged approach to RSI was 
implemented : 

First, the general and reciprocal procurement Memoranda 
of Understanding (MOUs) . . . were to be used to create a 
"two-way street" in defense trade. Each country would 
agree to remove domestic obstacles to the mutual 
penetration of defense markets. Each country would obtain 
exemptions from the other's customs, duties, tariffs, and 
protectionist legislation. In addition, each country 
would ideally give equal consideration to all qualified 
bidders regardless of the firm's nationality .... 

The second prong . . . was called "Dual Production, " and 
attempted to avoid redundant research and development 
(R&D) expenditures by encouraging nations to produce a 
system already developed by another member .... 

The third . . . strategy for achieving RSI was called 
the "Family of Weapons" approach .... it began 
collaboration at the R&D stage, requiring complete sharing 
of technology and know-how. It called for the 
complementary development of weapon systems in a 
particular mission area to be allocated to different 
countries, sequentially produced, and then made available 
to other members for production or co-production (Feldman, 

52) . 
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The first of these strategies, the negotiation of MOUs to 
aid in arms cooperation, received widespread support 
throughout NATO. Currently, formal agreements exist between 
the United States and 19 allied countries regarding arms 
cooperation. "Dual Production" continues to be favored by the 
DoD and the U. S. defense industry because "it approached 
standardization by relying on competitive R&D followed by 
licensed production or co-production" (Feldman, 52). The 
"Family of Weapons" approach was unpopular in America, on the 
industrial side because it called for relinquishing 
proprietary data (Feldman, 52) , and on the military side 
because of high level concern "over the flow of critical 
technology to the Eastern bloc" (Kitfield, "Finally," 77) . 

The Reagan administration approached arms cooperation with 
an entirely different attitude. Armaments cooperation 

. . . was pushed on all fronts. The "Leprosy Letter" 
was repudiated; the first international cooperative 
development program was started (Terminally Guided Warhead 
for the Multiple Launch Rocket System) ; defense 
cooperation offices were established in most major U. S. 
embassies to promote cooperation; and over thirty new 
cooperative development MOUs were signed (Naval, 24). 

Two other developments in RSI implementation during this 
period are the Nunn amendments to the FY85 and FY86 Defense 
Appropriations Authorization Act. The former professes the 
Government's support of RSI 

. . . but intends to achieve it by creating an open, 
competitive market in defense equipment- -a trading 
structure that is hoped will accomplish more than the 
"piecemeal" government MOUs of (the) past (Feldman, 53). 
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The latter takes a more proactive role by requiring all new 
weapon development programs be examined for possible 
international cooperation; by providing $100M as "seed money" 
for new, international codevelopment programs; and by 
providing $25M to allow the U. S. to test and evaluate systems 
already developed by Allies prior to beginning domestic 
production of similar systems (Naval, 26) . Congressional 
funding in support of cooperative developments climbed from 
$100M in FY86 , to $145M in FY87, to $150M in FY88, to $153M in 
FY89, before falling to $115M in FY90 (Cole, 61). Also in 
1987, the Quayle amendment permitted the DoD to cooperate with 
foreign nations from R&D through procurement (Naval, 25-26) . 

President Bush has generally continued the arms 
cooperation policies of his predecessor. The Bush 
Administration has 

. . . directed embassy personnel to increase the level 
of assistance provided to U. S. defense companies, created 
the Center for Defense Trade within the State Department, 
and proposed a "defense GATT" (General Agreement on Trade 
and Tariffs) that would allow free and open trade in arms 
and defense technology within the NATO Alliance, and with 
other U. S. Allies. In March 1991, the Administration 
proposed that the Export - Import Bank guarantee up to $1 
billion in commercial loans to members of NATO, Australia, 
Japan and Israel to purchase defense equipment from U. S. 
contractors (U. S. Congress, 21). 

On April 16, 1990, the President issued his policy on 
offset agreements. In essence, the policy prohibits the 
United States Government from participating in 

. . . any guarantees of offsets to any allied government 
or participating industry. This participation goes so far 
as to say that no agency can encourage, enter directly 
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into, or commit U. S. companies to any offset arrangements 
as they relate to any sale of defense articles or services 
to other governments. Additionally no U. S. funds may be 
used to finance offsets .... The new guidance does not 
prohibit negotiations for offsets by any government with 
the commercial industries involved as long as there is no 
U. S. Government encouragement, participation or 
commitment for the offset agreements which might be 
finally realized (McCarroll, 6-7). 

F. IMPACT OF RSI 

Objectively, the impact of the various RSI initiatives can 
best be appreciated by examining the balance in defense trade 
between the United States and the European allies. Most 
analysts agree that in the 1970s, the ratio of defense trade 
between the U. S. and its NATO allies was as much as 10:1 in 
favor of the United States (Costello, 70; Kitfield, 
"Obstacles," 86). Following Perry's RSI initiatives, this 
ratio was reduced to a level of 4.8:1 in favor of the U. S. in 
FY84 (Roos, 23) . The figures for FY86 show a further leveling 
off of the trade ratio, to 1.6:1 in favor of the United States 
(Kitfield, "Obstacles," 86; Roos, 23); this last ratio can be 
interpreted considering Nunn amendment. Other interpretations 
can be offered for the decline- - Kitfield points out that the 
FY84 figures are, in part at least, because the European 
nations reduced their defense purchases from the United States 
by 45 percent from the previous year (Kitfield, "Finally, " 
76) . However, Congressional calculations show that the 
transatlantic defense trade ratio, which in 1981 had reached 
a high of 12:1 in favor of the U. S., had only dropped to 
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about 5.2:1 in 1988 (U. S. Congress, 49). In any event, the 
overall trend proves that some effort has been expended in 
behalf of RSI. 

Another set of figures that reinforces the declining trade 

ratio balances concerns military contracts: 

The amount of U. S. military contracts awarded abroad 
nearly doubled (during the period FY80 to FY86) from $5.4 
billion to $9 billion .... During the same period, 
foreign military sales (FMS) by U. S. companies were cut 
in half, from $14.8 billion in 1980 to $7.1 billion (in 
1986) (Kitfield, "Obstacles," 86). 

G. OBSTACLES ON THE TWO-WAY STREET 

Although RSI is vigorously promoted on both sides of the 
Atlantic, very real obstacles impede optimal progress down the 
"two-way street." 

On this side of the Atlantic, a general problem is that 

there is not a clear, national consensus in support of RSI: 

RSI never became a "national decision, " its proponents 
having failed to win the blessing of Congress, the 
military services, industry, and labor .... Each group, 
for a variety of reasons, has at one time or another 
obstructed or resisted RSI (Feldman, 53) . 

Four major obstacles can be identified: 

• the "not invented here" syndrome; 

• economic protectionism; 

• technological isolationism; 

• the NATO burdensharing controversy (Feldman, 54). 



24 



The "not invented here" syndrome is primarily a military 
criticism of RSI which rests on three pillars. First, the 
Pentagon is rightfully concerned about any arms cooperative 
program that could "leave the U. S. dependent on a foreign 
source for any component of its defense needs" (Feldman, 54) . 
Secondly, the Pentagon reasons that "a variety of non- 
standardized defense systems would be more likely to thwart 
Soviet countermeasures than a few standardized systems" 
(Feldman, 54) . Finally, some officials fear that the quality 
of our forces' weapons might be sacrificed to achieve the 
political goal of armaments cooperation (Feldman, 54) . 

The obstacle of economic protectionism is a Congressional 
reaction to RSI. Although the MOUs were negotiated to ease 
the restrictions of protectionist statutes, elected officials 
can still be influenced by both industry and labor in their 
districts. For instance, the 

. . . Aerospace Industries Association (AIA) and the 
Electronic Industries Association (EIA) . . . have 
testified before Congress . . . that RSI is a form of 
"affirmative action" for the Europeans .... The 
implications for U. S. jobs are great, according to the 
AIA, which estimates that for every $1 billion in export 
sales lost, 70,000 U. S. jobs disappear as well (Feldman, 
55) . 

Constraining armaments cooperation, Congress has supported the 
Arms Export Control and the Export Administration Acts; has 
enacted the requirement that the Department of Commerce 
participate in the already slow MOU process; and has included 
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individual protectionist measures in most of the recent 
Defense Authorization Acts (Naval, 26). 

Perhaps the greatest obstacle on the two-way street is the 
issue of burdensharing, which arises from the perception that 
"some countries (in the NATO alliance) may be doing more, or 
not doing enough, in contributing to collective security" 
(Lightburn, 26) . The issue has often been couched in terms of 
each member's financial and military contribution to NATO 
expressed as a percentage of GNP/GDP. A recently completed 
study by NATO has explored the burdensharing problem. 

The study, done in 1988, recognized that there were many 
factors, besides financial and military, that comprised a 
country's contributions to the alliance, and that these 
factors could not all be quantified or compared (Lightburn, 
26). Nevertheless, an assessment for each country's total 
contribution was estimated and agreed upon by all other 
countries in the alliance (Lightburn, 27) . As a result of 
these assessments, proposals were formulated to maintain the 
quality of the allied defence, to redress some deficiencies 
that the assessments discovered, and to enhance security in 
the future (Lightburn, 27-28) : 

In the first of these categories, . . . countries were 
asked to improve their performance in meeting NATO goals, 
to enhance the forces which are committed to the Alliance, 
to improve force sustainability, to continue to support 
commonly funded programmes, to strive for improved 
armaments cooperation, to provide more assistance to 
developing defence industry nations and to search for 
greater efficiencies in the defence area. In the second 
category, . . . countries were asked to consider 
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improvements to the NATO Composite Force for North Norway, 
to the Allied Command Europe (ACE) Mobile Force and to the 
on-call naval force in the Mediterranean, to improve where 
possible host country support arrangements, to enhance 
reinforcement capabilities and to improve the Alliance's 
airborne early warning capability. The third category 
challenged countries to look at a better rationalization 
of existing defence arrangements, to examine and improve 
the potential of reserve and mobilizable capabilities, and 
to explore other ways of enhancing security through 
additional common or joint funding of requirements 
(Lightburn, 28) . 



H. DOD SUPPORT OF ARMAMENTS COOPERATIONS 

The Department of Defense has supported both Executive and 

Congressional efforts to further armaments cooperations. 

Directives are in place which form the basis of Defense 

Department policy on international armaments cooperation. 

Some key directives are listed below: 

• DoD Directive 2000.9, "International Co-Production 
Projects and Agreements Between the United States and 
Other Countries or International Organizations," dated 
January 23, 1974. 



• DoD Directive 3100.3, "Cooperation with Allies in Research 
and Development of Defense Equipment, " dated September 27, 
1963 . 



• DoD Directive 3100.4, "Harmonization of Qualitative 
Requirements for Defense Equipment of the United States 
and Allies," dated September 27, 1963. 



• DoD Directive 5530.3, "International Agreements," dated 
June 11, 1987. 



Directive 2000.9 is currently being revised; its revision 
would cancel the original 2000.9 and Directives 3100.3 and 
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3100.4, along with various memoranda issued by the Department. 
The draft revision proposes a ten-point DoD policy concerning 
international armaments cooperation; it says that it would be 
DoD policy that 

• The DoD would cooperate with U. S. Allies in the research 
and development, testing and evaluation (RDT&E) ; 
production; procurement; sale and follow-on support of 
conventional defense equipment when such cooperation 
promotes U. S. foreign policy objectives; U. S. security 
interests; the fulfillment of validated operational 
requirements; an approach for integrating DoD's 
requirement generation, acquisition management, and 
planning, programming and budgeting systems; and the 
maintenance or improvement of the U. S. defense industrial 
base . 

• The DoD would participate in cooperative programs that 
foster the collective security by encouraging participants 
to adopt interoperable equipment, compatible support 
arrangements, and common operational doctrine; by 
encouraging participants to invest appropriately in modern 
conventional defense; by making the most efficient use of 
the total scientific, technical, financial and industrial 
resources available to participants; and by fostering 
defense industrial capabilities in all participating 
countries to meet the military requirements of the 
alliance . 

• The DoD would maximize its resources available for defense 
of the alliance by conserving DoD resources through access 
to those foreign defense goods and services that provide 
cost-effective alternatives for meeting U. S. operational 
requirements; and by facilitating the common defense of 
the alliance through promoting equitable access to the 
most cost effective goods, technology and services, 
reducing duplicate efforts at RDT&E, production and 
support, and by fostering a more efficient market for 
defense goods. 

• The DoD would encourage consensus and validation of 
military requirements through treaties or agreements at 
the earliest practicable stage in the requirements' 
generation, and through all stages of the requirements' 
formulation. 

• The DoD would authorize the release of classified or 
unclassified U. S. information and technology if such 
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disclosure was subject to a Technology Assessment Control 
Plan and if such disclosure complied with law and other 
DoD Directives. 

• The DoD would protect and advance the U. S. industrial 
base in negotiating and implementing cooperative 
agreements by considering the effects of the agreements on 
the defense industrial base; and by interfacing with the 
Department of Commerce (DOC) regarding the implications of 
agreements on the industrial competitive position of U. S. 
industry. 

• The DoD would prepare an industrial base factors analysis 
for each agreement to assess the defense industrial 
implications of cooperative agreements. 

• The DoD would not encourage, enter directly into, or 
commit U. S. firms to any offset arrangement through a 
cooperative agreement except as directed by the President 
through the National Security Council. 

• The DoD would maintain a consistent approach to 
cooperative agreements by negotiating and concluding all 
agreements in accordance with DoD Directive 5530.3 or the 
Security Assistance Management Manual, as applicable. 

• The DoD would continually monitor and review cooperative 
programs to ensure consistency with U. S. interests and 
objectives, and would maintain a consolidated index of all 
cooperative agreements (Draft, 2-6). 



DoD Directive 5530.3 "International Agreements" 
establishes the requirements for, and restrictions on, 
authority to negotiate or conclude an international agreement. 
The directive defines "international agreement" broadly as any 
agreement concluded with one or more foreign governments that 
is signed or agreed to by personnel of the U. S. Government, 
that signifies the intention of the parties to be bound in 
international law, and that is identified as an international 
agreement or other similar language (2-1) . The definition 
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specifically excludes certain other official compacts made by 
Government personnel such as contracts made under the Federal 
Acquisition Regulations (FAR) and Foreign Military Sales (FMS) 
Credit Agreements (2-1). DoD Directive 5530.3 establishes 
standard procedures for requesting authority to negotiate or 
conclude international agreements, and assigns responsibility 
for that authority to the Under Secretary of Defense for 
Policy (USD(P)). For certain categories of agreements, the 
directive delegates authority to negotiate and conclude 
international agreements to the Service Secretaries, the 
Chairman of the Joint Chiefs of Staff (CJCS) , the Director of 
the Defense Security Assistance Agency (DSAA) , the Under 
Secretary of Defense for Acquisition (USD (A)), and others. 

I. DON SUPPORT OF ARMAMENTS COOPERATION 

In December, 1989, a Naval Research Advisory Committee 
(NRAC) Report on International Research and Development (NRAC 
Report No. 89-7) was completed. The committee panel was 
comprised of representatives from Government, private 
industry, and universities, and chaired by Mr. Gerald Cann, 
then of General Dynamics Corporation. The panel had been 
formed to address the following questions as they pertained to 
the Navy's international cooperative research and development 
(IR&D) programs: 

• How may the Navy enhance identification and assessment of 

foreign technologies to facilitate both the rapid and long 
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term incorporation of these technologies into naval weapon 
systems? 

• How may the Navy better identify potential cooperative 
development programs? What fundamental elements impact 
the success or failure of a cooperative development? 
Which elements should be considered in determining 
priority? 

• What role can industry assume to facilitate international 
cooperative programs and take advantage of available 
foreign technologies? What is necessary to maximize 
industry involvement in these programs and strengthen the 
U. S. industrial base? How may IR&D funding be used in 
these programs? 

• What incentives should be offered to promote international 
cooperative programs? How may the Navy reward program 
managers and industry for cost savings achieved through 
successful programs? 

• How may international programs be used to promote the 
maritime strategy objectives of the Navy? 

• How may the coordination and review of Navy international 
programs be improved? What organizational changes are 
necessary? 

• Are the fiscal goals established by the OSD for service 
participation in international programs, practical and 
adequate? 

• How will new legislation, requiring Department of Commerce 
review of all proposed international cooperative 
development Memoranda of Understanding (MOUs) , impact Navy 
programs and U. S. industry (Naval, 15-16) . 



An immediate cause for the panel's formation was the fact 
that two ongoing cooperative programs had been recently 
cancelled and several others appeared to be in trouble (Naval, 
3) . Factors underlying the cancellations were readily 
discernible. These included changes in the external 
environment in which IR&D is conducted; the lack of a formal 
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national policy on international armaments cooperation; and 
the means by which armaments cooperative agreements are 
impacted by actions of Congress, the Department of Defense and 
the Department of the Navy (Naval, 3). 

The panel recognized several trends in the external 
environment that impact on the Navy's IR&D efforts, central of 
which is the increasing " globalization of the economy . 
industry and trade. This, in turn, has fostered increasing 
interdependence among nations " (Naval, 21) . The panel 
acknowledged an erosion in the United States' traditional 

leadership position in several important 
(technological) areas, such as microelectronics, ceramics 
and manufacturing process technology. Foreign capability 
in the design of integrated circuits, launch vehicles and 
commercial aircraft underscores the general trend toward 
technology leveling . As a result, intense competitive 
pressure has promoted widespread dual use of our most 
advanced technologies (Naval, 21). 

A final environmental trend identified by the panel was 
the fact that our focus on the threats facing our nation was 
shifting from a military threat by the Soviet bloc to an 
economic threat by the other industrialized nations of the 
world, specifically the Pacific Rim nations and Western Europe 
(Naval, 21). The panel attributed industry's narrow 

. . . preoccupation with short term profit and corporate 
book value (with putting) us at a distinct disadvantage in 
dealing with Japan's longer goal of securing market share 
(Naval, 21) . 

Concerning the Government's national policy on 
international armaments cooperation, the panel was quick to 
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point out that the Government has no formal national policy 
even though 

. . . armaments cooperation with North Atlantic Treaty 

Organization (NATO) and other friendly countries has been 
an integral part of DoD's RDT&E and production procurement 
programs since World War II (Naval, 23). 

Instead of a formal, written policy on international armaments 

cooperation, an ad hoc policy evolved "in accordance with the 

prevailing international views held by each administration" 

(Naval, 23) . 

The panel recognized that the Department of the Navy has 
generally followed OSD direction for international armaments 
cooperation by issuing SECNAV instructions that direct 
compliance with the relevant DoD Directives and by generating 
its own set of OPNAV instructions to provide guidelines for 
conducting MOU negotiations (Naval, 29) . The NRAC panel did 
level some criticism at DoN for not being sufficiently 
sensitive to U. S. industrial base concerns and for the ultra- 
conservative position the Navy has traditionally taken 
regarding the licensing of its equipment for use and sales 
overseas (Naval, 30) . 

J. THE NAVY INTERNATIONAL PROGRAMS OFFICE (IPO) 

Coordination of the Navy's international armaments 
cooperative efforts is accomplished by the International 
Programs Office (IPO) , reporting to the Deputy Assistant 
Secretary of the Navy for International Policy (DASN(IP)) . In 
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IPO, the Director for International Agreements is responsible 
for oversight of the acquisition MOUs, for assisting in the 
development and negotiations of MOUs by the Program Managers, 
and for policy coordination. 

The IPO office traditionally, at regular intervals, issued 
a "call" for Candidate Nomination Proposals (CNPs) throughout 
the various Program Offices of the Navy. Offices whose 
programs were fully funded typically declined to submit CNPs 
while offices facing shortfalls and budget cuts proposed 
programs for MOU consideration. 

Recent procedures established by DoD Instruction 5000.2, 
"Defense Acquisition Management Policies and Procedures, " may 
lead to an increase in both the quantity and quality of 
candidate programs. The instruction requires that a report be 
submitted to Congress annually regarding international 
cooperative efforts. The report, the Cooperative Research and 
Development Projects Report, is prepared by the Deputy Under 
Secretary of Defense for International Programs (DUSD(IP)), 
and reviews Acquisition Category I (ACAT I) , Major Defense 
Acquisition Programs, activity for the year (ll-D-1-3). The 
report will provide Congress with two pieces of information: 

• A description of status, funding and schedule of existing 
cooperative research and development projects for which a 
Memorandum of Understanding (or other formal agreement) 
has been entered into; 

• A description of the purpose, funding and schedule of any 
proposed new projects included in the President's budget 
proposal for which a Memorandum of Understanding (or other 
formal agreement) has not been entered into (ll-D-1-3). 
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The instruction also requires that a Cooperative Opportunities 
Document be included as Annex G to the Integrated Program 
Summary document as part of the Defense Acquisition Board 
(DAB) Milestones Review procedures (ll-C-3) . This requirement 
applies to all Acquisition Categories programs at Milestones 
I through IV; the Cooperative Opportunities Document would 
address the following areas: 

• Are there any similar projects in development or 
production by one or more major allies of the United 
States? 

• If yes, could that project satisfy, or be modified in 
scope so as to satisfy, the military requirements of the 
United States? 

• What are the advantages and disadvantages of seeking to 
structure a cooperative development program with one or 
more Allied nations? 

• What alternate forms of cooperation could be appropriate 
for this project (DoD Manual 5000. 2M, 4-H-l-l and -2)? 

The IP Office also targets likely MOU prospects through the 
annual Planning, Programming and Budgeting System (PPBS) 
review process, during which the office focuses primarily on 
mid-level programs (ACAT II programs, the lower range of ACAT 
I programs, and the upper range of ACAT III programs) . 

Once the IP Office identifies a candidate cooperative 
program, either through a successful PPBS targeting or a call 
for candidate programs, the office discusses the program with 
the originator to assess the program's potential as a 
cooperative agreement. For programs with significant 
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potential, the originator drafts and submits a Project 
Nomination Proposal (PNP) to the IPO. The IPO then performs 
the "quick look" evaluation that is the subject of this 
research; those proposals evaluated likely to succeed are 
forwarded for notification to the Deputy Assistant Secretary 
of the Navy for International Policy (DASN(IP)) . The DASN(IP) 
either sends the PNP back to IPO with feedback or submits the 
PNP to the Assistant Secretary of the Navy for Research, 
Development and Acquisition (ASN(RDSA)) for approval to 
Request Authority to Negotiate (RAN) . Once ASN(RD&A) approval 
is obtained, the IPO submits the PNP to the USD (A) for 
authority to negotiate in accordance with DoD Directive 
5530.3 . 
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III. PRESENTATION OF DATA 



A. BACKGROUND TO THE DEVELOPMENT OF SUCCESS CRITERIA 
1. Original Evaluation Procedure 

The IP Office originally used the following 20-point 
checklist to evaluate Project Nomination Proposals: 

• MOU REQUIREMENT. Why must a MOU be employed? Is use of 
a MOU based on legal or policy grounds? Would use of 
another international program mechanism (e.g., FMS LOA, 
export license, etc.) achieve the same objectives? 

• LEGAL. If a MOU is required, which statute (s) provide 
authority for the MOU? More specifically, what is the 
statutory authority for each of the obligations undertaken 
by the U. S. Government in the MOU? 

• NUMBER OF PARTICIPANTS. How many nations desire to 
participate in the MOU? (Note that in general, bilateral 
agreements are encouraged.) 

• MOU SCHEDULE. Can authority to negotiate be obtained and 
can a MOU be negotiated and concluded in time to meet 
desired program schedule (s) ? 

• PROGRAM OBJECTIVE. Is the objective of the MOU clearly 
stated? What is the motivation and benefit to the U. S. 
Government? What is the perceived motivation and benefit 
to the foreign government (s) ? (Note that the combination 
of the two must create a "win-win" situation in order for 
the program to succeed.) 

• PROGRAM REQUIREMENT. Is there a documented U. S. 
requirement which has: 

- OPNAV support and/or approval? 

- CINC/JCS support and/or approval? 

- SYSCOM/Lab support and/or approval? 

- RD&A support and/or approval? 

- OSD support and/or approval? 

- Congressional support and/or approval? 
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• Is the combined U. S. -foreign requirement clearly stated 
in the MOU, and does it realistically reflect the needs of 
all parties? 

• PROGRAM SCOPE OF WORK. Are proposed program tasks and 
associated schedules clearly stated? Will they achieve 
program objectives and meet program requirements in the 
areas of technical performance, cost and schedule? 

• MANAGEMENT. Will this be a U. S. managed project, a 
foreign country managed project, or a jointly managed 
project? What is the management hierarchy (steering 
groups, program offices, etc.)? Will the program 
office (s) be integrated or not? 

• FINANCIAL. What is the total cost of the project 
(financial and non- financial) ? What is the U. S. share 
(financial/non- financial) versus the foreign share? What 
is the United States/foreign cost share rationale 
(production off-take, equal share, etc.), and are the 
contributions versus benefits equitable? What are the 
U. S. sources of funds/non- financial contributions? If 
DoN funds, are they in the budget/POM? If Nunn funds, 
have they been approved by DoN/OSD? If NRC waiver (s) are 
proposed, have they been approved by DoN/OSD and/or 
foreign governments involved? 

• CURRENCY. Will the program be managed in U. S. dollars, 
foreign currency, or a mix? Will the establishment of a 
joint program bank account be required? 

• CONTRACTUAL ARRANGEMENTS. Will U. S. contractual 
procedures be used or not? Who will serve as the U. S. 
contracting agency (if applicable)? What type of contract 
(FFP, CPIF, etc.) will be employed? Are there any special 
arrangements or procedures? If so, are they in accordance 
with the FAR? 

• WORK- SHARE. If a cooperative MOU, what is the anticipated 
division of work- share among participants, by volume and 
by type (high tech, low tech) ? Does the proposed work- 
share align with cost-share (i. e. is it equitable)? 

• OFFSET AGREEMENTS . Are there industrial offset agreements 
addressed in the MOU that are associated with this 
program? Will the U. S. Government play a role? Is that 
role consistent with DoD policy on offsets? 

• INDUSTRIAL BASE IMPACT. What will be the impact on the 
U. S. industrial base (including sub- tier suppliers) of 
proposed work- share/of f set agreements? 
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• INTELLECTUAL PROPERTY RIGHTS. What background and 

foreground data rights are anticipated to be transferred 
to foreign participants? Do U. S. rights align with U. S. 
contributions? (Note that for substantial U. S. 

contributions, we should obtain a greater degree of data 
rights than our prospective partners.) 

• TECHNOLOGY TRANSFER. Are sensitive technologies to be 
shared or transferred? Have such transfers been cleared 
through DoN and OSD (DTSA) ? Is TTSARB approval required? 
Has disclosure guidance to the negotiating team been 
developed and issued? Has a Delegation of Disclosure 
Letter (DDL) to the Program Manager (PM) been issued? 

• DISCLOSURE OF CLASSIFIED INFORMATION. Will U. S. 
classified information or material be disclosed to foreign 
nationals during the course of the program? Is approval 
by the National Disclosure Policy Committee (NDPC) 
required? (Note that both technology transfer and 
disclosure of CMI should be addressed in the MOU 
negotiating guidance and DDL.) 

• THIRD PARTY SALES AND TRANSFERS. Will future U. S. 
disclosures require concurrence of other participants? 
(Note that if the U. S. possesses considerable "background 
equity" in the program, permitting foreign "veto" of 
potential U. S. sales should be avoided.) 

• MOU CLARITY. Does the MOU clearly state the arrangements 
listed above or are they to be documented by side 
agreements? 

• REQUIRED DOCUMENTATION. Have the following supporting 

documents been prepared: Legal Memorandum; Fiscal 

Memorandum, Technology Security Risk Assessment (or its 
successor, Technology Assessment and Control Plan) ; 
Industrial Base Factors Analysis? 



The IP Office halted use of the checklist since it offered an 
entirely subjective overview and since it could not provide a 
methodology for comparing one PNP to another. 
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2. OSD Decision Matrix 



The IP Office considered adopting the following four-part 
decision matrix used by the Office of the Secretary of Defense 
for selecting programs to pursue among those nominated for 
international research and development projects: 



SUITABILITY FOR 
COLLABORATION 


HIGH (9 PTS ) 
RED 


MEDIUM (4 
PTS ) YELLOW 


LOW (1 PT) 
GREEN 


1. STATUS OF MOU 


SIGNED 


IN PROCESS 


PROCESS NOT 
YET STARTED 


2. LIKELIHOOD OF 


CONSISTENT 


REQUIRES 


REQUIRES 


FOREIGN SUPPORT OR 


WITH FOREIGN 


SOME CHANGE 


RADICAL DE- 


COMMITMENT 


APPROACHES 
TO DEFENSE 


IN APPROACH 


PARTURE IN 
APPROACH OR 
NEW POLICY 


3 . SIMILARITY WITH 


INCORPO- 


INCORPO- 


REQUIRES 


ONGOING PROJECT 


RATES MAJOR 


RATES SOME 


TERMINATION 




ONGOING 


ELEMENTS OF 


OR REDUCTION 




PROJECT 


ONGOING 

RDT&E 


OF ONGOING 
PROJECT 


4. LIKELIHOOD OF 


U. S. AND 


SIMILAR 


VERY 


SUCCESS IN 


ALLIES ARE 


SCHEDULE 


DIFFERENT 


COLLABORATION 
A. TIMING 


ON SAME 
SCHEDULE 




SCHEDULES 


B. COSTS 


COULD 


MODERATE 


LITTLE 




RADICALLY 
REDUCE COST 


INFLUENCE 


IMPACT 


C. TECHNOLOGY 


POTENTIAL 


POTENTIAL 


LITTLE 


SHARING 


FOR SIGNIF- 


FOR MODERATE 


POTENTIAL 




ICANT 

SAVINGS 


SAVINGS 


FOR SAVINGS 


D. SECURITY 


UNCLASSIFIED 


CONFIDENTIAL 


SECRET; VERY 




NO SENSITIVE 


MODERATELY 


SENSITIVE 




TECHNOLOGY 


SENSITIVE 


TECHNOLOGY 




OR SYSTEMS 


TECHNOLOGY 
OR SYSTEMS 


OR SYSTEMS 
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SUITABILITY FOR 
COLLABORATION 


HIGH (9 PTS ) 
RED 


MEDIUM (4 
PTS) YELLOW 


LOW (1 PT) 
GREEN 


E. RATIONALIZATION, 
S TAND ARD I Z AT ION, 
INTEROPERABILITY 
(RSI) 


SUPPORTS 
AREAS WHERE 
RSI IS 
CRUCIAL 


SUPPORTS 
AREAS WHERE 
RSI IS MOD- 
ERATELY 
IMPORTANT 


SUPPORTS 
AREA WHERE 
RSI IS NOT 
VERY 

IMPORTANT 


5. DEFENSE INDUS- 
TRIAL BASE IMPACT 

A. ACCESS TO UNIQUE 
FOREIGN ASSETS 


PROVIDES 
SIGNIFICANT 
ACCESS TO 
KEY FOREIGN 
ASSETS 


MODERATE 

ACCESS 


LITTLE OR NO 
ACCESS 


B. ACCESS TO UNIQUE 
U. S. ASSETS 


PROVIDES 
LITTLE OR NO 
ACCESS TO 
KEY 

U. S. ASSETS 


MODERATE 

ACCESS 


SIGNIFICANT 

ACCESS 


C. IMPACT ON U. S. 
PRODUCTION LEVELS 
( <5 YEARS) 


SUPPORTS 
EFFICIENT 
U. S. 

PRODUCTION 

LEVELS 


LITTLE OR NO 
IMPACT 


UNDERCUTS 
U. S. 

PRODUCTION 

LEVELS 


D. IMPACT ON U. S. 
PRODUCTION LEVELS 
( >5 YEARS) 


SUPPORTS 
EFFICIENT 
U. S. 

PRODUCTION 

LEVELS 


LITTLE OR NO 
IMPACT 


UNDERCUTS 
U. S. 

PRODUCTION 

LEVELS 


E. IMPACT ON U. S. 
TECHNOLOGICAL LEAD 


ENHANCES 
U. S. LEAD 
IN MANTECH 


LITTLE OR NO 
IMPACT 


DEGRADES 
U. S. LEAD 
IN MANTECH 



POTENTIAL TO RE- 
DRESS CONVENTIONAL 
FORCE DEFICIENCY 


HIGH (9 PTS) 
RED 


MEDIUM (4 
PTS) YELLOW 


LOW (1 PT) | 

GREEN 


1. NEW CONVENTIONAL 
DEFENSE CAPABILITY 


GIVES CAPA- 
BILITY TO 
REDRESS 
SHORT TERM 
IMBALANCE 


REDRESSES 
. LONG TERM 
IMBALANCE 


PROVIDES 
INCREMENTAL 
IMPROVEMENT 
IN NATO/ 
ALLIED CAPA- 








BILITY 


2. MEETS U. S. 

OPERATIONAL 

REQUIREMENT 


SIGNED JCS 
AND SERVICE 
REQUIREMENT 


REQUIREMENT 
IN REVIEW 


NO FORMAL 

REQUIREMENT 

EXISTS 
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POTENTIAL TO RE- 
DRESS CONVENTIONAL 
FORCE DEFICIENCY 


HIGH (9 PTS ) 
RED 


MEDIUM (4 
PTS) YELLOW 


LOW (1 PT) 
GREEN 


3 . ADDRESSES FORMAL 
NATO/ALLIED DEFI- 
CIENCY 


FORMALLY 

APPROVED 

REQUIREMENT 


LINKED TO 
DEFICIENCIES 
IN NATO CMF 
OR OTHER 
FORMAL 
DOCUMENT 


WEAK LINKAGE 
TO FORMAL 
DOCUMENT 



PRIORITY WITHIN 
SERVICE/AGENCY 


HIGH (9 PTS) 
RED 


MEDIUM (4 
PTS) YELLOW 


LOW (1 PT) 
GREEN 


i 1. LIKELIHOOD OF 
FOLLOW-ON 
ACQUISITION 


MEETS VERY 
HIGH PRIORI- 
TY SERVICE/ 
AGENCY 
OBJECTIVE 


MEETS MODER- 
ATE PRIORITY 
OBJECTIVE 


GENERALLY 
SUPPORTIVE 
OF OBJEC- 
TIVES 


2. PLANNED COMMIT- 
MENT TO PROCUREMENT 


SOLIDLY IN 
LONG TERM 
PLANS 


MODERATE 
LEVEL OF 
LONG TERM 
COMMITMENT 


NOT YET 
INCORPORATED 
IN LONG TERM 
PLANS 


3. WILLINGNESS TO 
FUND "PRE-MOU" 
COSTS 


SERVICE/ 
AGENCY WILL 
COVER ALL 
SUCH COSTS 


SERVICE/ 
AGENCY WILL 
COVER ONLY 
PART OF 
COSTS 


SERVICE/ 
AGENCY WILL 
NOT COVER 
SUCH COSTS 


4. STATUS OF U. S. 
PROJECT 


WELL 

ESTABLISHED 


— 


NEW 


5. WILLINGNESS TO 
USE U. S. 
FACILITIES 


WILLING TO 
COMMIT KEY 
FACILITIES 


MAY COMMIT 
KEY 

FACILITIES 


NOT WILLING 
TO COMMIT 
KEY FACILI- 
TIES 



SUPPORT IN FIVE- 
YEAR DEFENSE PLAN 


HIGH (9 PTS) 
RED 


MEDIUM (4 
PTS) YELLOW 


LOW (1 PT) 
GREEN 


1. SUPPORT IN 
SERVICE POM 


CONTAINED IN 
SERVICE POM 


COMMITMENT 
TO PUT IN 
POM LATER 


NO 

COMMITMENT 

YET 


2 . SUPPORT IN 
FOREIGN BUDGET 


CONTAINED IN 

FOREIGN 

BUDGET 


COMMITMENT 
TO PUT IN 
FOREIGN 
BUDGET 


NO 

COMMITMENT 

YET 
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Though the OSD evaluation format did allow for a 
quantitative comparison of one candidate program to another, 
the IP Office decided not to adopt the OSD format because it 
did not adequately consider the point of view of the foreign 
participants in the MOU. Their experience with MOUs convinced 
the IP Office that the key to the success of an MOU was the 
"degree of convergence" between the U. S. decision factors and 
the foreign decision factors; a high degree of convergence 
indicated a synergism at work in the MOU that helped ensure a 
successful program. 

B. INITIAL SUCCESS CRITERIA EVALUATION 

1. U. S. /Foreign Evaluation Criteria 

On October 5, 1990, the IP Office developed the following 
weighted criteria by which a candidate program could be 
evaluated: 



I. LEVEL OF 
COMMITMENT : 


10 


5 


1 


-CONGRESS/ 

EQUIVALENT 


LEGISLATION 


MEMBER/ STAFF 


NO INTEREST 


-OSD/MOD 


USD (A) UP 


DUSD/ASD UP 


WORKING 

LEVEL 


-SERVICE/ 

EQUIVALENT 


ASN(RDA) UP 


SPONSOR 

SYSCOM/PEO 


WORKING 

LEVEL 
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i II. PROGRAMMATIC 
FACTORS 


10 


5 


1 


- PROGRAM 


MAJOR SYSTEM 


OTHER SYSTEM 


TECHBASE 


OBJECTIVE 


FLEET 


FLEET 


AND/OR 




DELIVERY 


DELIVERY 


STUDIES 


-MILITARY 

REQUIREMENTS 


APPROVED OR 


APPROVED TOR 


INTENT/ NO 
OR REQUIRED 


-ALLIANCE 


APPROVED 


APPROVED 


INTENT TO 


REQUIREMENTS 


NATO OUTLINE 
STAFF TARGET 


NATO MISSION 
NEED 

EVALUATION 


DEVELOP 


-MOU/ PROGRAM 


NO MOU/ 


CONSTRAINED 


MAJOR 


SCHEDULES 


PROGRAM 


MOU/ PROGRAM 


SCHEDULE 




SCHEDULE 

PROBLEMS 


SCHEDULES 


DISCONNECT 


-INDUSTRIAL BASE 


ENHANCES 


NO IMPACT ON 


NEGATIVE 


IMPACT 


U. S. 


U. S. 


U. S. 




INDUSTRIAL 


INDUSTRIAL 


INDUSTRIAL 




BASE 


BASE 


BASE IMPACT 


-3RD PARTY SALES 


DESIRE BROAD 


DESIRE 


DESIRE NO 


OBJECTIVES 


SALES 


SELECTED 

SALES 


SALES 



III. ABILITY TO 


10 


5 


1 


CONTRIBUTE 








-FUNDING 


FULLY FUNDED 


PARTIALLY 


INTENT TO 




IN POM 


FUNDED IN 
POM 


POM 


-GOVERNMENT 


FULL SYSCOM/ 


PARTIAL 


SYS COM/ LAB 


TECHNOLOGY 


LAB SUPPORT 


SYS COM/ LAB 


RESOURCE 






SUPPORT 


PROBLEMS 


- INDUSTRY 


FULL INDUS- 


PARTIAL 


INDUSTRIAL 


TECHNOLOGY 


TRY SUPPORT/ 


SUPPORT/ 


SUPPORT/ 




TECHNOLOGY 


TECHNOLOGY 


TECHNOLOGY 




AVAILABLE 


AVAILABLE 


PROBLEMS 


-TECHNOLOGY 


NO TECH 


SELECTED 


MAJOR TECH 


TRANSFER 


TRANSFER 


TECH TRANS- 


TRANSFER 




ISSUES 


FER ISSUES 


ISSUES 


-SECURITY 


LOW RISK 


MEDIUM RISK 


HIGH RISK 
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Five-step Evaluation Method 



2 . 



Using the above criteria, the IP office conducted the 
following five- step evaluation process on each Project 
Nomination Proposal: 

• Establish initial U. S. "Strength Rating" based on 
evaluation of proposed program vis-a-vis IP "program 
success" indicators. 

• Analyze Foreign Intentions/Capabilities in a similar 
manner. 

• Compare results of the first two steps to "match up" U. 
S. /Foreign strengths/weaknesses. Since U. S. and foreign 
factors can either support, neutralize, or conflict with 
each other, the initial U. S. "Strength Rating" may be 
positively or negatively affected by the degree of 
convergence between the U. S. and Foreign intentions and 
capabilities . 

• Qualitatively determine the proposed program's 
"Probability of Success" and "Payoff Potential" if program 
is successful. 

• Develop a consolidated priority listing which takes into 
account the program's quantitative "Composite Strength 
Rating, " and qualitative "Probability of Success" and 
"Payoff Potential" assessments. 



The individual criteria for each area (Level of 
Commitment, Programmatic Factors, Ability to Contribute) were 
assigned values from one to ten; an average for each area was 
then determined and multiplied by a predetermined weight 
(0.75, 1.5, and 1.0 for Level of Commitment, Programmatic 
Factors, and Ability to Contribute, respectively) to arrive at 
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an area "Strength Rating." Area strength ratings for both the 
United States and the foreign participants were compared, 
summed and multiplied by a second weight to arrive at a 
composite area strength rating. This second weight (ranging 
in multiples of 0.25 from 0.50 to 1.50) was subjectively 
determined by the IP Office by aligning the individual 
criteria under each U. S. area with their foreign 
counterparts. If a positive degree of convergence existed for 
an area, the weight would be a value greater than one. A 
value of one was assigned for a neutral degree of convergence, 
and a value less than one was assigned for a negative degree 
of convergence. These composite area strength ratings were 
then summed to determine the final composite value by which 
programs would be ranked. 

3. Initial Application Of Evaluation Criteria and Method 
a. Programs Evaluated 

The following programs were initially evaluated; the short 
titles in parentheses are for cross-referencing purposes: 
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PROGRAM TITLE 


FOREIGN PARTICIPANTS 


NATO MECHANICAL 
MINESWEEPING SYSTEM 
(NMMS ) 


FRANCE , NETHERLANDS , 
SPAIN, ITALY 


ANTI -SHIP MISSILE 

COUNTERMEASURES 

(ASMCM) 


UNITED KINGDOM 


"SPINNAKER" (SPIN) 


CANADA 


U. S. PACOM COMBINED 
INTEROPERABILITY PROGRAM 
(CIP) 


AUSTRALIA, SOUTH KOREA, 
JAPAN 


NATO CSNI ( CSNI ) 


UNITED KINGDOM, FRANCE, 
NETHERLANDS , CANADA 


CONTINUOUS PROCESSING OF 
SOLID PROPELLANTS 
(C2P2 ) 


FRANCE 



Jb. Evaluation Sheets 

The individual evaluation sheets for the above programs 
are recreated on the following six pages. Although the format 
of these recreations differs slightly from the originals due 
to software limitations, the data and general sense of the 
evaluations remain true to the originals. 



47 



PROGRAM SHORT TITLE: NMMS 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


3 


-MOD 


6 


-Service 


8 


-Service/Equiv. 


9 


-Strength Rating 


3.00 


-Strength Rating 


4.00 


Degree 


of Convergence 


Weight: 1.0 




Programmatic Factors (1.5) 


Programmatic Factors 


(1.5) 


-Program Objective 


4 


-Program Objective 


4 


-Military Reqm' ts 


10 


-Military Reqm'ts 


10 


-Alliance Reqm'ts 


10 


-Alliance Reqm'ts 


10 


-MOU/ Program 


5 


-MOU/ Program 


5 


Schedules 




Schedules 




-Industrial Base 


5 


-Industrial Base 


5 


Impact 




Impact 




-3rd Party Sales 


5 


-3rd Party Sales 


5 


Objectives 




Obj ectives 




-Strength Rating 


9.75 


-Strength Rating 


9.75 


Degree 


of Convergence 


Weight: 1.25 




Abilitv to Contribute (1.0) Abilitv to Contribute 


(1.0) 


- Funding 


10 


- Funding 


10 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


8 


- Industry 


8 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


5 


-Strength Rating 


9.50 


-Strength Rating 


8.60 


Degree 


of Convergence 


Weight: 1.5 




U. S. Strength 


22.25 


Foreign Strength 


22.35 



Probability of Success: Medium-High 

Payoff Potential: Medium-High 



COMPOSITE STRENGTH 58.53 
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PROGRAM SHORT TITLE; ASMCM 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0 . 75) 


Level of Commitment 


(0.75) 


- Congress 


1 


- Legislature 


1 


-OSD 


1 


-MOD 


3 


-Service 


6 


- Service/Equiv . 


5 


-Strength Rating 


2 . 00 


-Strength Rating 


2.25 


Degree 


of Convergence 


Weight: 1.0 




Programmatic Factors (1.5) 


Programmatic Factors 


(1.5) 


-Program Objective 


7 


-Program Objective 


7 


-Military Reqm'ts 


10 


-Military Reqm'ts 


5 


-Alliance Reqm'ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/Program 


10 


-MOU/Program 


10 


Schedules 
-Industrial Base 


5 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Objectives 
-Strength Rating 


12 . 00 


Obj ectives 
-Strength Rating 


10 . 13 


Degree 


of Convergence 


Weight: 1.25 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


7 


- Funding 


7 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


NA 


- Industry 


NA 


Technology 




Technology 




-Technology 


5 


-Technology 


5 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


7.33 


-Strength Rating 


8.00 


Degree 


of Convergence 


Weight: 1.5 




U. S. Strength 


21.33 


Foreign Strength 


20.38 



Probability of Success: High-High 

Payoff Potential: High-High 

COMPOSITE STRENGTH 54.91 
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PROGRAM SHORT TITLE: SPIN 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


3 


-MOD 


5 


-Service 


8 


-Service/Equiv. 


8 


-Strength Rating 


3 . 00 


-Strength Rating 


3.50 


Degree 


of Convergence 


Weight: 1.0 




Proarammatic Factors (1.5) 


Proarammatic Factors 


(1.5) 


-Program Objective 


6 


-Program Objective 


6 


-Military Reqm' ts 


10 


-Military Reqm'ts 


10 


-Alliance Reqm' ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/Program 


9 


-MOU/Program 


9 


Schedules 
-Industrial Base 


5 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Objectives 
-Strength Rating 


11.25 


Objectives 
-Strength Rating 


11.25 


Degree 


of Convergence 


Weight: 1.25 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


10 


- Funding 


10 


-Government 


8 


-Government 


8 


Technology 




Technology 




- Industry 


5 


- Industry 


5 


Technology 




Technology 




-Technology 


7 


-Technology 


7 


Transfer 




Transfer 




- Security 


NA 


-Security 


10 


-Strength Rating 


7.50 


-Strength Rating 


8 . 00 


Degree 


of Convergence 


Weight: 1.25 




U. S. Strength 


21.75 


Foreign Strength 


22.75 



Probability of Success: Medium-High 



Payoff Potential: High-High 

COMPOSITE STRENGTH 54 . 00 
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PROGRAM SHORT TITLE 

UNITED STATES 


: CIP 


FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


- Legislature 


1 


-OSD 


8 


-MOD 


8 


-Service 


7 


-Service/Equiv. 


9 


-Strength Rating 


4 . 00 


-Strength Rating 


4 . 50 


Degree 


of Convergence 


Weight: 1.0 




Procrrammatic Factors (1.5) 


Proqrammatic Factors 


(1.5) 


-Program Objective 


5 


-Program Objective 


5 


-Military Reqm' ts 


8 


-Military Reqm'ts 


8 


-Alliance Reqm'ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/ Prog ram 


10 


-MOU/Program 


10 


Schedules 
-Industrial Base 


5 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Obj ectives 
-Strength Rating 


10 .50 


Obj ectives 
-Strength Rating 


10.50 


Degree 


of Convergence 


Weight: 1.25 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


5 


- Funding 


9 


-Government 


8 


-Government 


8 


Technology 




Technology 




- Industry 


NA 


- Industry 


NA 


Technology 




Technology 




-Technology 


8 


-Technology 


8 


Transfer 




Transfer 




-Security 


NA 


- Security 


10 


-Strength Rating 


7 . 00 


-Strength Rating 


8 . 75 


Degree 


of Convergence 


Weight: 1.0 




U. S. Strength 


21.50 


Foreign Strength 


23 . 75 



Probability of Success: Medium-High 

Payoff Potential: Medium-High 

COMPOSITE STRENGTH 50.50 
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PROGRAM SHORT TITLE: CSNI 



UNITED STATES 


(0.75) 


FOREIGN 


(0.75) 


Level of Commitment 


Level of Commitment 


- Congress 


3 


- Legislature 


2 


-OSD 


4 


-MOD 


5 


-Service 


6 


- Service/Eguiv . 


6 


-Strength Rating 


3.25 


-Strength Rating 


3.25 


Degree 


of Convergence 


Weight: 1.0 





Programmatic Factors (1.5) Programmatic Factors (1.5) 



-Program Objective 


1 


-Program Objective 


3 


-Military Reqm' ts 


2 


-Military Reqm'ts 


3 


-Alliance Reqm'ts 


10 


-Alliance Reqm'ts 


10 


-MOU/Program 


8 


-MOU/Program 


8 


Schedules 




Schedules 




-Industrial Base 


5 


-Industrial Base 


5 


Impact 




Impact 




-3rd Party Sales 


NA 


-3rd Party Sales 


NA 


Obj ectives 




Obj ectives 




-Strength Rating 


7.80 


-Strength Rating 


8.70 


Degree 


of Convergence 


Weight: 1.25 




Abilitv to Contribute (1.0) Abilitv to Contribute 


(1.0) 


- Funding 


5 


- Funding 


10 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


NA 


- Industry 


NA 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


- Security 


5 


-Strength Rating 


8.33 


-Strength Rating 


8 . 75 


Degree 


of Convergence 


Weight: 1.25 




U. S. Strength 


19.38 


Foreign Strength 


20.70 



Probability of Success: Medium-High 

Payoff Potential: Medium-Medium 

COMPOSITE STRENGTH 48.48 
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PROGRAM SHORT TITLE: C2P2 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


- Congress 


1 


-Legislature 


1 


-OSD 


2 


-MOD 


4 


-Service 


4 


- Service/Equiv . 


6 


-Strength Rating 


1.75 


-Strength Rating 


2 . 75 


Degree 


of Convergence 


Weight: 1.0 




Proarammatic Factors 


i (1.5) 


Proarammatic Factors 


(1.5) 


-Program Objective 


1 


-Program Objective 


1 


-Military Reqm' ts 


1 


-Military Reqm' ts 


1 


-Alliance Reqm' ts 


NA 


-Alliance Reqm' ts 


NA 


-MOU/Program 


6 


-MOU/Program 


6 


Schedules 
-Industrial Base 


10 


Schedules 
-Industrial Base 


10 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Objectives 
-Strength Rating 


6 . 75 


Objectives 
-Strength Rating 


6 . 75 


Degree 


of Convergence 


Weight: 1.0 





Ability to Contribute (1.0) Ability to Contribuce (1.0) 



- Funding 


2 


- Funding 


8 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


7 


- Industry 


7 


Technology 




Technology 




-Technology 


8 


-Technology 


8 


Transfer 




Transfer 




-Security 


NA 


-Security 


5 


-Strength Rating 


6.75 


-Strength Rating 


7.60 


Degree 


of Convergence 


Weight: 1.25 




U. S. Strength 


15.25 


Foreign Strength 


17.10 



Probability of Success: Medium-Medium 



Payoff Potential: Medium-High 

COMPOSITE STRENGTH 35.94 
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c. Results of Initial Application 
The following table summarizes the results of this initial 
evaluation: 



PROGRAM 

NAME 


COMPOSITE 

STRENGTH 

RATING 


U. S. 

STRENGTH 

RATING 


FOREIGN 

STRENGTH 

RATING 


PROBABIL- 
ITY OF 
SUCCESS 


PAYOFF 

POTEN- 

TIAL 


NMMS 


58.53 


22.25 


22.35 


MEDIUM- 

HIGH 


HIGH 


ASMCM 


54.91 


21.33 


20.38 


HIGH 


HIGH 


SPIN 


54 . 00 


21.75 


22 . 75 


MEDIUM- 

HIGH 


HIGH 


CIP 


54 . #0 


21.50 


23.75 


MEDIUM- 

HIGH 


MEDIUM- 

HIGH 


CSNI 


48 .48 


19 . 38 


20.70 


MEDIUM- 

HIGH 


MEDIUM 


C2P2 


35.94 


15.25 


17.10 


MEDIUM 


MEDIUM- 

HIGH 



4. Second Application of Evaluation Criteria and Method 
a. Programs Evaluated 

An additional 12 programs were evaluated later in the same 
fiscal year. These programs are listed below: 
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PROGRAM TITLE 


FOREIGN PARTICIPANTS 


REMOTE SENSING UNIT WITH 
SYNTHETIC APERTURE RADAR 
(RSU) 


NORWAY , 

UNITED KINGDOM 


AIRBORNE ELECTROMAGNETIC 
SENSOR (AES) 


CANADA 


TACTICAL LIGHTWEIGHT 
SURVEILLANCE SATELLITE 
(TLSS ) 


AUSTRALIA 


TACTICAL TARGET SUPPORT 
(TTS ) 


AUSTRALIA 


DISTRIBUTED TACTICAL 
SURVEILLANCE (DTS ) 


UNITED KINGDOM 


TACTICAL SHIP 
SURVEILLANCE (TSS) 


UNITED KINGDOM 


OCEANOGRAPHY COOPERATION 
(OC-J) 


JAPAN 


OCEANOGRAPHY COOPERATION 
(OC-N) 


NORWAY 


OCEAN COLOR ASSESSMENT, 
G-I-N SEA (OCA-G) 


UNITED KINGDOM, NORWAY, 
ICELAND, DENMARK 


OCEAN COLOR ASSESSMENT, 
MEDITERRANEAN SEA 
(OCA-K) 


UNITED KINGDOM, NORWAY, 
ICELAND, DENMARK 


MILLIMETER WAVE SEA 
SURVEILLANCE (MWSS) 


GERMANY, UNITED KINGDOM 


REMOTE SPECTROMETRY 
RESEARCH (RSR) 


CANADA, UNITED KINGDOM 



b. Evaluation Sheets 

The following pages replicate the evaluation sheets 
resulting from this second application of the success criteria 
and 5 -step evaluation method to candidate programs: 
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PROGRAM SHORT TITLE; RSU 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


8 


- Service/Equiv . 


8 


-Strength Rating 


2 .50 


-Strength Rating 


2 . 50 


Degree 


of Convergence 


Weight: 1.0 




Proarammatic Factors (1.5) 


Proarammatic Factors 


(1.5) 


-Program Objective 


5 


-Program Objective 


5 


-Military Reqm'ts 


10 


-Military Reqm'ts 


10 


-Alliance Reqm'ts 


5 


-Alliance Reqm'ts 


5 


-MOU/ Program 


7 


-MOU/ Program 


7 


Schedules 
-Industrial Base 


5 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Obj ectives 
-Strength Rating 


9 . 60 


Obj ectives 
-Strength Rating 


9.60 


Degree 


of Convergence 


Weight: 1.25 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


7 


- Funding 


5 


-Government 


9 


-Government 


9 


Technology 




Technology 




- Industry 


10 


- Industry 


10 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


9.00 


-Strength Rating 


8.80 


Degree 


of Convergence 


Weight: 1.25 




U. S. Strength 


21.10 


Foreign Strength 


20.90 



Probability of Success: Medium-High 

Payoff Potential: Medium-High 



COMPOSITE STRENGTH 51.25 
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PROGRAM SHORT TITLE: AES 



UNITED STATES 


(0.75) 


FOREIGN 


(0.75) 


Level of Commitment 


Level of Commitment 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


4 


-Service/Equiv. 


5 


-Strength Rating 


1.50 


-Strength Rating 


1.75 


Degree 


of Convergence 


Weight: 1.0 





Programmatic Factors (1.5) Programmatic Factors (1.5) 



-Program Objective 


3 


-Program Objective 


3 


-Military Reqm'ts 


3 


-Military Reqm'ts 


3 


-Alliance Reqm'ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/ Program 


10 


-MOU/ Program 


10 


Schedules 




Schedules 




-Industrial Base 


4 


-Industrial Base 


4 


Impact 




Impact 




-3rd Party Sales 


NA 


-3rd Party Sales 


NA 


Obj ectives 




Obj ectives 




-Strength Rating 


7.50 


-Strength Rating 


7.50 


Degree 


of Convergence 


Weight: 1.25 




Abilitv to Contribute (1.0) Abilitv to Contribute 


(1.0) 


- Funding 


7 


- Funding 


7 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


10 


- Industry 


10 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


9.25 


-Strength Rating 


9.40 


Degree 


of Convergence 


Weight: 1.25 




U. S. Strength 


18.25 


Foreign Strength 


18 . 65 



Probability of Success: Medium-High 



Payoff Potential: Medium 



COMPOSITE STRENGTH 45.31 
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PROGRAM SHORT TITLE: TLSS 



UNITED STATES 


(0.75) 


FOREIGN 


(0.75) 


Level of Commitment 


Level of Commitment 


-Congress 


1 


- Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


5 


-Service/Equiv. 


5 


-Strength Rating 


1 . 75 


-Strength Rating 


1.75 


Degree 


of Convergence 


Weight: 1.0 





Programmatic Factors (1.5) Programmatic Factors (1.5) 



-Program Objective 


7 


-Program Objective 


7 


-Military Reqm' ts 


1 


-Military Reqm'ts 


1 


-Alliance Reqm'ts 


7 


-Alliance Reqm'ts 


7 


-MOU/ Program 


10 


-MOU/ Program 


10 


Schedules 




Schedules 




-Industrial Base 


7 


-Industrial Base 


4 


Impact 




Impact 




-3rd Party Sales 


NA 


-3rd Party Sales 


NA 


Objectives 




Obj ectives 




-Strength Rating 


9 . 60 


-Strength Rating 


8 . 70 


Degree 


of Convergence 


Weight: 1.25 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


2 


- Funding 


2 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


10 


- Industry 


4 


Technology 




Technology 




-Technology 


5 


-Technology 


5 


Transfer 




Transfer 




- Security 


NA 


-Security 


10 


-Strength Rating 


6.75 


-Strength Rating 


6.20 


Degree 


of Convergence 


Weight: 1.25 




U. S. Strength 


18.10 


Foreign Strength 


16.65 



Probability of Success: Medium- High 

Payoff Potential: Medium-High 

COMPOSITE STRENGTH 42.56 
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PROGRAM SHORT TITLE: TTS 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


5 


-Service/Equiv. 


5 


-Strength Rating 


1.75 


-Strength Rating 


1.75 


Degree 


of Convergence 


Weight: 1.0 




Programmatic Factors (1.5) 


Programmatic Factors 


(1.5) 


-Program Objective 


5 


-Program Objective 


5 


-Military Reqm' ts 


2 


-Military Reqm'ts 


2 


-Alliance Reqm' ts 


7 


-Alliance Reqm'ts 


7 


-MOU/ Program 


8 


-MOU/ Program 


8 


Schedules 
-Industrial Base 


6 


Schedules 
-Industrial Base 


6 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Objectives 
-Strength Rating 


8.40 


Obj ectives 
-Strength Rating 


8.40 


Degree 


of Convergence 


Weight: 1.25 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


1 


- Funding 


1 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


10 


- Industry 


10 


Technology 




Technology 




-Technology 


5 


-Technology 


5 


Transfer 




Transfer 




- Security 


NA 


-Security 


10 


-Strength Rating 


6.50 


-Strength Rating 


7.20 


Degree 


of Convergence 


Weight: 1.25 




U. S. Strength 


16.65 


Foreign Strength 


17.35 



Probability of Success: 
Payoff Potential: 



Medium- High 
Medium 



COMPOSITE STRENGTH 41.63 
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PROGRAM SHORT TITLE; DTS 



UNITED STATES 


(0.75) 


FOREIGN 


(0.75) 


Level of Commitment 


Level of Commitment 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


- Service 


5 


-Service/Equiv. 


5 


-Strength Rating 


1.75 


-Strength Rating 


1.75 


Degree 


of Convergence 


Weight: 1.0 





Programmatic Factors (1.5) Programmatic Factors (1.5) 



-Program Objective 


2 


-Program Objective 


2 


-Military Reqm' ts 


10 


-Military Reqm'ts 


10 


-Alliance Reqm' ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/ Program 


7 


-MOU/ Program 


7 


Schedules 




Schedules 




-Industrial Base 


5 


-Industrial Base 


5 


Impact 




Impact 




-3rd Party Sales 


NA 


-3rd Party Sales 


NA 


Objectives 




Obj ectives 




-Strength Rating 


9.00 


-Strength Rating 


9.00 


Degree 


of Convergence 


Weight: 1.00 




Abilitv to Contribute (1.0) Abilitv to Contribute 


(1.0) 


- Funding 


5 


- Funding 


5 


-Government 


8 


-Government 


8 


Technology 




Technology 




- Industry 


5 


- Industry 


5 


Technology 




Technology 




- Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


7.00 


-Strength Rating 


7.60 


Degree 


of Convergence 


Weight: 1.00 




U. S. Strength 


17.75 


Foreign Strength 


18.35 



Probability of Success: Medium 



Payoff Potential: Medium 

COMPOSITE STRENGTH 36.10 
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PROGRAM SHORT TITLE: TSS 



UNITED STATES 


(0.75) 


FOREIGN 


JO. 75) 


Level of Commitment 


Level of Commitment 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


5 


-Service/Equiv. 


5 


-Strength Rating 


1.75 


-Strength Rating 


1.75 


Degree 


of Convergence 


Weight: 1.0 





Programmatic Factors (1.5) Programmatic Factors (1.5) 



-Program Objective 


1 


-Program Objective 


1 


-Military Reqm'ts 


10 


-Military Reqm'ts 


10 


-Alliance Reqm'ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/Program 


8 


-MOU/Program 


8 


Schedules 




Schedules 




-Industrial Base 


5 


-Industrial Base 


5 


Impact 




Impact 




-3rd Party Sales 


NA 


-3rd Party Sales 


NA 


Obj ectives 




Objectives 




-Strength Rating 


9 . 00 


-Strength Rating 


9 . 00 


Degree 


of Convergence 


Weight: 1.00 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


6 


- Funding 


4 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


5 


- Industry 


5 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


7.75 


-Strength Rating 


7.80 


Degree 


of Convergence 


Weight: 0.75 




U. S. Strength 


18.50 


Foreign Strength 


18.55 



Probability of Success: Medium 



Payoff Potential: Medium-Low 

COMPOSITE STRENGTH 33 . 16 
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PROGRAM SHORT TITLE: OC-J 



UNITED STATES 


(0.75) 


FOREIGN 


(0.75) 


Level of Commitment 


Level of Commitment 


-Congress 


1 


- Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


5 


-Service/Equiv. 


1 


-Strength Rating 


1.75 


-Strength Rating 


0.75 


Degree 


of Convergence Weight: 1.0 




Programmatic Factors (1.5) 


Programmatic Factors 


(1.5) 


-Program Objective 


2 


-Program Objective 


2 


-Military Reqm' ts 


3 


-Military Reqm' ts 


3 


-Alliance Reqm' ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/ Program 


10 


-MOU/ Program 


10 


Schedules 




Schedules 




-Industrial Base 


5 


-Industrial Base 


5 


Impact 




Impact 




-3rd Party Sales 


NA 


-3rd Party Sales 


NA 


Obj ectives 




Obj ectives 




-Strength Rating 


7.50 


-Strength Rating 


7.50 


Degree 


of Convergence Weight: 1.00 




Abilitv to Contribute (1.0) 


Abilitv to Contribute 


(1.0) 


- Funding 


1 


- Funding 


1 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


NA 


- Industry 


NA 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


7.00 


-Strength Rating 


7.75 


Degree 


of Convergence Weight: 1.00 




U. S. Strength 


16.25 


Foreign Strength 


16.00 



Probability of Success: Medium 

Payoff Potential: Medium 

COMPOSITE STRENGTH 32.25 
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PROGRAM SHORT TITLE: OC-N 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


5 


-Service/Equiv. 


1 


-Strength Rating 


1 . 75 


-Strength Rating 


0 . 75 


Degree 


of Convergence 


Weight: 1.0 




Proarammatic Factors (1.5) 


Programmatic Factors 


(1.5) 


-Program Objective 


2 


-Program Objective 


2 


-Military Reqm' ts 


3 


-Military Reqm'ts 


3 


-Alliance Reqm'ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/ Program 


10 


-MOU/Program 


10 


Schedules 
-Industrial Base 


5 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Obj ectives 
-Strength Rating 


7.50 


Obj ectives 
-Strength Rating 


7.50 


Degree 


of Convergence 


Weight: 1.00 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


1 


- Funding 


1 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


NA 


- Industry 


NA 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


7.00 


-Strength Rating 


7.75 


Degree 


of Convergence 


Weight: 1.00 




U. S. Strength 


16.25 


Foreign Strength 


16.00 



Probability of Success: Medium 

Payoff Potential: Medium 

COMPOSITE STRENGTH 32.25 
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PROGRAM SHORT TITLE: OCA-G 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0. 75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


4 


-Service/Eguiv. 


1 


-Strength Rating 


1.50 


-Strength Rating 


0.75 


Degree 


of Convergence 


Weight: 1.0 




Proarammatic Factors (1.5) 


Proarammatic Factors 


(1.5) 


-Program Objective 


1 


-Program Objective 


1 


-Military Reqm'ts 


2 


-Military Reqm'ts 


2 


-Alliance Reqm'ts 


2 


-Alliance Reqm'ts 


2 


-MOU/ Prog ram 


10 


-MOU/ Prog ram 


10 


Schedules 
-Industrial Base 


5 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Obj ectives 
-Strength Rating 


6 . 00 


Obj ectives 
-Strength Rating 


6.00 


Degree 


of Convergence 


Weight: 1.00 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


6 


- Funding 


5 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


NA 


- Industry 


NA 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


8.67 


-Strength Rating 


8 . 75 


Degree 


of Convergence 


Weight: 1.00 




U. S. Strength 


16 . 17 


Foreign Strength 


15.50 



Probability of Success: Medium 



Payoff Potential: Medium-Low 

COMPOSITE STRENGTH 31.67 
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PROGRAM SHORT TITLE; OCA-M 



UNITED STATES 


(0.75) 


FOREIGN 


(0.75) 


Level of Commitment 


Level of Commitment 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


4 


-Service/Equiv. 


1 


-Strength Rating 


1.50 


-Strength Rating 


0.75 


Degree 


of Convergence 


Weight: 1.0 





Programmatic Factors (1.5) Programmatic Factors (1.5) 



- Program Ob j ect ive 


1 


-Program Objective 


l 


-Military Reqm' ts 


2 


-Military Reqm'ts 


2 


-Alliance Reqm'ts 


2 


-Alliance Reqm'ts 


2 


- MOU/Program 


10 


- MOU/Program 


10 


Schedules 




Schedules 




-Industrial Base 


5 


-Industrial Base 


5 


Impact 




Impact 




-3rd Party Sales 


NA 


-3rd Party Sales 


NA 


Obj ect ives 




Obj ect ives 




-Strength Rating 


6.00 


-Strength Rating 


6.00 


Degree 


of Convergence 


Weight: 1.00 




Abilitv to Contribute (1.0) Abilitv to Contribute 


(1.0) 


- Funding 


6 


- Funding 


5 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


NA 


- Industry 


NA 


Technology 




Technology 




-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


NA 


-Security 


10 


-Strength Rating 


8.67 


-Strength Rating 


8 . 75 


Degree 


of Convergence 


Weight: 1.00 




U. S. Strength 


16 . 17 


Foreign Strength 


15.50 



Probability of Success: Medium 

Payoff Potential: Medium-Low 

COMPOSITE STRENGTH 31.67 
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PROGRAM SHORT TITLE: MWSS 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


- Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


5 


-Service/Equiv. 


5 


-Strength Rating 


1.75 


-Strength Rating 


1 . 75 


Degree 


of Convergence 


Weight: 1.0 




Programmatic Factors (1.5) 


Programmatic Factors 


(1.5) 


-Program Objective 


1 


-Program Objective 


1 


-Military Reqm' ts 


4 


-Military Reqm' ts 


4 


-Alliance Reqm' ts 


NA 


-Alliance Reqm' ts 


NA 


-MOU/ Program 


10 


-MOU/ Program 


10 


Schedules 
-Industrial Base 


8 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Obj ectives 
-Strength Rating 


8.63 


Obj ectives 
-Strength Rating 


7.50 


Degree 


of Convergence 


Weight: 1.00 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


1 


- Funding 


1 


-Government 


10 


-Government 


10 


Technology 




Technology 




- Industry 


8 


- Industry 


8 


Technology 




Technology 




-Technology 


5 


-Technology 


5 


Transfer 




Transfer 




-Security 


NA 


-Security 


5 


-Strength Rating 


6.00 


-Strength Rating 


5 . 80 


Degree 


of Convergence 


Weight: 1.00 




U. S. Strength 


16.38 


Foreign Strength 


15.05 



Probability of Success: Medium 

Payoff Potential: Medium-Low 

COMPOSITE STRENGTH 31.43 
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PROGRAM SHORT TITLE: RSR 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


1 


-MOD 


1 


-Service 


5 


- Service /Equiv . 


5 


-Strength Rating 


1.75 


-Strength Rating 


1.75 


Degree 


of Convergence 


Weight: 1.0 




Programmatic Factors (1.5) 


Programmatic Factors 




-Program Objective 


1 


-Program Objective 


1 


-Military Reqm'ts 


1 


-Military Reqm'ts 


1 


-Alliance Reqm'ts 


NA 


-Alliance Reqm'ts 


NA 


-MOU/ Program 


10 


-MOU/ Program 


10 


Schedules 
-Industrial Base 


5 


Schedules 
-Industrial Base 


5 


Impact 

-3rd Party Sales 


NA 


Impact 

-3rd Party Sales 


NA 


Obj ectives 
-Strength Rating 


6.38 


Obj ectives 
-Strength Rating 


6 . 38 


Degree 


of Convergence 


Weight: 1.00 





Ability to Contribute (1.0) Ability to Contribute (1.0) 



- Funding 


1 


- Funding 


2 


-Government 


8 


-Government 


8 


Technology 




Technology 




- Industry 


5 


- Industry 


5 


Technology 




Technology 




-Technology 


5 


-Technology 


5 


Transfer 




Transfer 




- Security 


NA 


- Security 


10 


-Strength Rating 


4.75 


-Strength Rating 


6.00 


Degree 


of Convergence 


Weight: 1.00 




U. S. Strength 


12 . 88 


Foreign Strength 


14 . 13 



Probability of Success: Medium 

Payoff Potential: Medium 

COMPOSITE STRENGTH 27.00 
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c. Results of Second Application 



This second set of program evaluations is summarized in 
the table below: 



PROGRAM 

NAME 


COMPOSITE 

STRENGTH 

RATING 


U. S. 

STRENGTH 

RATING 


FOREIGN 

STRENGTH 

RATING 


PROBABIL- 
ITY OF 
SUCCESS 


PAYOFF 

POTEN- 

TIAL 


RSU 


51.25 


21 . 10 


20.90 


MEDIUM- 

HIGH 


MEDIUM- 

HIGH 


AES 


45.31 


18.25 


16.65 


MEDIUM- 

HIGH 


MEDIUM 


TLSS 


42.56 


18 . 10 


16 . 65 


MEDIUM- 

HIGH 


MEDIUM- 

HIGH 


TTS 


41.63 


16.65 


17.35 


MEDIUM- 

HIGH 


MEDIUM 


DTS 


36 . 10 


17 . 75 


18.35 


MEDIUM 


MEDIUM 


TSS 


33 . 16 


18 . 50 


18 . 55 


MEDIUM 


MEDIUM- 

LOW 


0C- J 


32.25 


16.25 


16 . 00 


MEDIUM 


MEDIUM 


OC-N 


32.25 


16.25 


16 . 00 


MEDIUM 


MEDIUM 


OCA-G 


31.67 


16 . 17 


11 . 5$ 


MEDIUM 


MEDIUM- 

LOW 


OCA-M 


31.67 


16 . 17 


15 . 50 


MEDIUM 


MEDIUM- 

LOW 


MWSS 


31.43 


16 .38 


15 . 05 


MEDIUM 


MEDIUM- 

LOW 


RSR 


27.00 


12 . 88 


14.13 


MEDIUM 


MEDIUM- 

LOW 
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C. REVISED SUCCESS CRITERIA EVALUATION 

1. Revised U. S. /Foreign Evaluation Criteria 

On July 30, 1991, the IP office revised the evaluation 

criteria. The revision separated two criteria ("Technology 
Transfer" and "Security") from the "Ability to Contribute" 
area and established them in a new area, "Technology/Security 
Factors." Under the "Programmatic Factors" area, the revision 
deleted one criterion ("Alliance Requirements"), replaced two 
criteria ("Industrial Base Impact" and "3rd Party Sales 
Objectives") with new ones ("Workshare" and "Payoff"), and 
added NATO operational requirements criteria to the "Military 
Requirements" section. The revised worksheet is provided 
below: 



I. LEVEL OF 
COMMITMENT : 


10 


5 


1 


-CONGRESS/ 

EQUIVALENT 


LEGISLATION 


MEMBER/STAFF 


NO INTEREST 


-OSD/MOD 


USD (A) UP 


DUSD/ASD UP 


WORKING 

LEVEL 


-SERVICE/ 

EQUIVALENT 


ASN(RDA) UP 


SPONSOR 

SYSCOM/PEO 


WORKING 

LEVEL 
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II. PROGRAMMATIC 
FACTORS 


10 


5 


1 


- PROGRAM 
OBJECTIVE 


MAJOR SYSTEM 

FLEET 

DELIVERY 


OTHER SYSTEM 

FLEET 

DELIVERY 


TECKBASE/ 

STUDIES 


-MILITARY 

REQUIREMENTS 


APPROVED 

OR/NSR 


APPROVED 

TOR/NST 


INTENT/NO 
OR REQUIRED 


-MOU/ PROGRAM 
SCHEDULES 


NO MOU/ 
PROGRAM 
SCHEDULE 
PROBLEMS 


CONSTRAINED 
MOU/ PROGRAM 
SCHEDULE 


MAJOR 

SCHEDULE 

DISCONNECT 


-WORKSHARE 


WORKSHARE 

EQUALS 

COSTSHARE 


WORKSHARE 
W/ IN PLUS OR 
MINUS 10% OF 
COSTSHARE 


WORKSHARE 
DOES NOT 
EQUAL 
COSTSHARE 


- PAYOFF 


GREATLY 

ENHANCES 

FLEET 

EFFECTIVE- 

NESS 


ENHANCES 

FLEET 

EFFECTIVE- 

NESS 


REDUNDANT 

SYSTEM 



III. ABILITY TO 
CONTRIBUTE 


10 


5 


1 


-FUNDING 


FULLY FUNDED 
IN POM 


PARTIALLY 
FUNDED IN 
POM 


INTENT TO 
POM 


- GOVERNMENT 
TECHNOLOGY 


FULL SYSCOM/ 
LAB SUPPORT 


PARTIAL 
SYS COM/ LAB 
SUPPORT 


SYS COM/ LAB 

RESOURCE 

PROBLEMS 



IV. TECHNOLOGY/ 
SECURITY 
FACTORS 


10 


5 


1 


-TECHNOLOGY 

TRANSFER 


NO TECH 

TRANSFER 

ISSUES 


SELECTED 
TECH TRANS- 
FER ISSUES 


MAJOR TECH 

TRANSFER 

ISSUES 


-SECURITY 


LOW RISK 


MEDIUM RISK 


HIGH RISK 
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2 . Revised Evaluation Method 

The evaluation process associated with this new set of 
criteria differs from the initial process. The weights (0.75, 
1.50, and 1.00) of the initial process are again assigned to 
the areas of "Level of Commitment, " "Programmatic Factors, " 
and "Ability to Contribute," and a weight of 1.00 is assigned 
to the new area "Technology/Security Factors." U. S. and 
foreign strength ratings are computed as before, but the 
composite strength rating computation differs. Under the 
initial process, each area's strength ratings (U. S. and 
foreign) were first summed and then multiplied by subjectively 
assigned weights; the resultant products were then summed to 
arrive at the composite strength rating. Under the revised 
procedure, all area strength ratings (both U. S. and foreign) 
are summed and multiplied by a single, subjectively assigned 
weight factor. Summary U. S. and foreign strength ratings are 
not computed under the revised process; neither are 
qualitative measures given to "Probability of Success" or 
"Payoff Potential." 

3. Application of Revised Evaluation Criteria and Method 
a. Programs Evaluated 

Four programs were evaluated using the revised criteria 
and procedures. The following table identifies these 
programs : 
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PROGRAM TITLE 


FOREIGN PARTICIPANTS 


MARITIME UNMANNED AERIAL 
VEHICLE (MUAV) 


CANADA, FRANCE, GERMANY, 
NETHERLANDS, NORWAY, 
UNITED KINGDOM 


ADVANCED STEEL SYSTEMS 
(ADV ST) 


JAPAN 


ALUMINUM OXYGEN BATTERIES 
(AL BAT) 


CANADA 


REMOTE ATMOSPHERIC AND 
IONOSPHERIC DETECTION 
SYSTEM (RAIDS) 


AUSTRALIA, BELGIUM, 
CANADA, GERMANY, SWEDEN 



b. Evaluation Sheets 

The evaluation sheets representing these four programs are 
reproduced on the following pages: 
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PROGRAM SHORT TITLE; MUAV 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


0 


-OSD 


1 


-MOD 


1 


-Service 


6 


-Service/Equiv. 


6 


-Strength Rating 


2 . 00 


-Strength Rating 


2 . 63 


Proqrammatic Factors 


(1.5) 


Proqrammatic Factors 


(1. 51 


-Program Objective 


8 


-Program Objective 


8 


-Military Reqm'ts 


8 


-Military Reqm'ts 


7 


-MOU/Program 


10 


-MOU/Program 


10 


Schedules 




Schedules 




-Workshare 


5 


-Workshare 


5 


- Payoff 


8 


- Payoff 


8 


-Strength Rating 


11.70 


-Strength Rating 


11.40 


Abilitv to Contribute (1.0) 


Ability to Contribute 


(1.0) 


- Funding 


10 


- Funding 


5 


-Government 


0 


-Government 


0 


Technology 




Technology 




-Strength Rating 


10.00 


-Strength Rating 


5.00 


Technoloav/Securitv 


(1.0) 


Technoloqv/Securitv 


(1.0) 


-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


10 


- Security 


0 


-Strength Rating 


10 . 00 


-Strength Rating 


10 . 00 



OVERALL STRENGTH MATCHING 1.00 

COMPOSITE STRENGTH 62.73 
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PROGRAM SHORT TITLE: ADV ST 



UNITED STATES 




Level of Commitment 


(0.75) 


- Congress 


1 


-OSD 


1 


-Service 


3 


-Strength Rating 


1 . 75 



Programmatic Factors (1.5) 



-Program Objective 6 
-Military Reqm'ts 3 
-MOU/ Program 2 

Schedules 

-Workshare 0 

-Payoff 8 

-Strength Rating 7.13 

Ability to Contribute (1.0) 

-Funding 5 

-Government 10 

Technology 

-Strength Rating 7.50 

Technology/Securitv (1.0) 

-Technology 9 

Transfer 

-Security 10 

-Strength Rating 9.50 



FOREIGN 



Level of Commitment (0.75) 



-Legislature 


1 


-MOD 


5 


- Service/Equiv . 


0 


-Strength Rating 


2.25 


Programmatic Factors 


(1.5) 


-Program Objective 


5 


-Military Reqm'ts 


0 


-MOU/Program 


0 


Schedules 




-Workshare 


0 


- Payoff 


7 


-Strength Rating 


9.00 



Ability to Contribute (1.0) 



- Funding 


0 


-Government 


8 


Technology 




-Strength Rating 


8 . 00 


Technology/Securitv 


(1.0) 


-Technology 


9 


Transfer 




-Security 


0 


-Strength Rating 


9 . 00 



OVERALL STRENGTH MATCHING 1.25 

COMPOSITE STRENGTH 58.00 
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PROGRAM SHORT TITLE: AL BAT 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


3 


-MOD 


1 


-Service 


5 


-Service/Equiv. 


1 


-Strength Rating 


2.25 


-Strength Rating 


0 . 75 


Programmatic Factors 


(1.5) 


Programmatic Factors 


(1,5) 


-Program Objective 


5 


-Program Objective 


0 


-Military Reqm'ts 


10 


-Military Reqm'ts 


0 


-MOU/ Program 


5 


-MOU/ Program 


5 


Schedules 




Schedules 




-Workshare 


0 


-Workshare 


2 


- Payoff 


9 


- Payoff 


2 


-Strength Rating 


10.88 


-Strength Rating 


4.50 


Abilitv to Contribute (1.0) 


Ability to Contribute 


(1.0) 


- Funding 


10 


- Funding 


7 


-Government 


10 


-Government 


0 


Technology 




Technology 




-Strength Rating 


10 . 00 


-Strength Rating 


7.00 


Technoloqv/Security 


(1-0) 


Technology /Securi tv 


(1-0) 


-Technology 


10 


-Technology 


10 


Transfer 




Transfer 




-Security 


0 


-Security 


10 


-Strength Rating 


10.00 


-Strength Rating 


10 . 00 



OVERALL STRENGTH MATCHING 1.00 

COMPOSITE STRENGTH 55.38 
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PROGRAM SHORT TITLE: RAIDS 



UNITED STATES 
Level of Commitment 


(0.75) 


-Congress 


1 


-OSD 


1 


-Service 


2 


-Strength Rating 


1.00 


Programmatic Factors 


(1.5) 


-Program Objective 


2 


-Military Reqm' ts 


8 


-MOU/ Prog ram 
Schedules 


8 


-Workshare 


0 


- Payoff 


4 


-Strength Rating 


8.25 



Ability to Contribute (1.0) 



- Funding 


5 


-Government 


10 


Technology 
-Strength Rating 


7.50 


Technologv/Securitv 


(1.0) 


-Technology 


5 


Transfer 




-Security 


5 


-Strength Rating 


5.00 



FOREIGN 



Level of Commitment (0.75) 



- Legislature 


1 


-MOD 


1 


- Service/Equiv. 


1 


-Strength Rating 


0.75 



Programmatic Factors (1.5) 



-Program Objective 5 



-Military Reqm' ts 0 
- MOU / P r og r am 0 

Schedules 

-Workshare 0 

-Payoff 2 

-Strength Rating 5.25 

Ability to Contribute (1.0) 

- Funding 3 

-Government 10 

Technology 

-Strength Rating 6.50 

Technoloav/Securitv (1.0) 

-Technology 5 

Transfer 

-Security 0 

-Strength Rating 5.00 



OVERALL STRENGTH MATCHING 1.00 

COMPOS I TE S TRENGTH 39.25 
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c. Re-evaluation of Programs Using Revised Evaluation 
Criteria and Method 

Attempting to standardize the evaluation process, the IP 
office re-evaluated the six programs initially evaluated 
(NMMS, ASMCM, SPIN, CIP, CSNI , and C2P2) using the new 
criteria and procedures. These re-evaluations are reproduced 
on the following pages: 
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PROGRAM SHORT TITLE: NMMS 



UNITED STATES 



Level of Commitment (0.75) 



- Congress 


1 


-OSD 


3 


- Service 


8 


-Strength Rating 


3.00 


Programmatic Factors 


(1.5) 


-Program Objective 


4 


-Military Reqm'ts 


10 


-MOU/ Program 


5 


Schedules 




-Workshare 


5 


- Payoff 


8 


-Strength Rating 


9 . 60 



Ability to Contribute (1.0) 



- Funding 


10 


-Government 


10 


Technology 




-Strength Rating 


10.00 


Technologv/Securitv 


(1.0) 


-Technology 


10 


Transfer 




-Security 


5 


-Strength Rating 


7.50 



FOREIGN 



Level of Commitment (0.75) 



- Legislature 


1 


-MOD 


6 


-Service/Equiv. 


9 


-Strength Rating 


4.00 



Programmatic Factors (1.5) 



-Program Objective 


4 


-Military Reqm'ts 


10 


-MOU/Program 


10 


Schedules 




-Workshare 


5 


- Payoff 


8 


-Strength Rating 


11 . 1C 



Ability to Contribute (1.0) 



- Funding 


10 


-Government 


10 


Technology 
-Strength Rating 


10.00 


Technologv/Securitv 


(1.0) 


-Technology 


10 


Transfer 




-Security 


0 


-Strength Rating 


10.00 



OVERALL STRENGTH MATCHING 1.50 

COMPOSITE STRENGTH 75.20 
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PROGRAM SHORT TITLE: ASMCM 



UNITED STATES 
Level of Commitment 


(0.75) 


-Congress 


1 


-OSD 


1 


-Service 


6 


-Strength Rating 


2.00 


Programmatic Factors 


(1.5) 


-Program Objective 


7 


-Military Reqm'ts 


10 


-MOU/ Program 
Schedules 


10 


-Workshare 


5 


- Payoff 


10 


-Strength Rating 


12 . 60 



Ability to Contribute (1.0) 



-Funding 


7 


-Government 


10 


Technology 




-Strength Rating 


8.50 


Technologv/Security 


(1.0) 


-Technology 


5 


Transfer 




-Security 


10 


-Strength Rating 


5.00 



FOREIGN 



Level of Commitment (0.75) 



- Legislature 


1 


-MOD 


3 


-Service/Equiv. 


5 


-Strength Rating 


2.25 


Programmatic Factors 


(1.5) 


- Program Ob j ect ive 


7 


-Military Reqm'ts 


5 


-MOU/Program 


10 


Schedules 




-Workshare 


5 


- Payoff 


10 


-Strength Rating 


11 . 10 



Ability to Contribute (1.0) 



-Funding 


7 


-Government 


10 


Technology 




-Strength Rating 


8.50 


Technology /Security 


(1-0) 



-Technology 5 

Transfer 

-Security 0 

-Strength Rating 5.00 



OVERALL STRENGTH MATCHING 1.50 

COMPOSITE STRENGTH 65.95 
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PROGRAM SHORT TITLE; SPIN 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


- Legislature 


1 


-OSD 


3 


-MOD 


5 


-Service 


8 


-Service/Equiv. 


8 


-Strength Rating 


3.00 


-Strength Rating 


3.50 


Proqrammatic Factors 


(1.5) 


Proqrammatic Factors 


(1.5) 


-Program Objective 


6 


-Program Objective 


6 


-Military Reqm' ts 


10 


-Military Reqm' ts 


10 


-MOU/ Program 


9 


-MOU/ Program 


9 


Schedules 




Schedules 




-Workshare 


5 


-Workshare 


5 


-Payoff 


10 


- Payoff 


10 


-Strength Rating 


12.00 


-Strength Rating 


12.00 


Abilitv to Contribute (1.0) 


Abilitv to Contribute 


(1.0) 


- Funding 


10 


- Funding 


10 


-Government 


8 


-Government 


8 


Technology 




Technology 




-Strength Rating 


9.00 


-Strength Rating 


9 . 00 


Technoloqv/ Securitv 


(1.0) 


Technoloqv/Securitv 


(1.0) 


-Technology 


7 


-Technology 


7 


Transfer 




Transfer 




-Security 


10 


-Security 


0 


-Strength Rating 


8.50 


-Strength Rating 


7.00 



OVERALL STRENGTH MATCHING 1.25 

COMPOSITE STRENGTH 68.50 
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PROGRAM SHORT TITLE: CIP 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


8 


-MOD 


8 


-Service 


7 


-Service/Equiv. 


9 


-Strength Rating 


4.00 


-Strength Rating 


4.50 


Proqrammatic Factors 


(1.5) 


Proqrammatic Factors 


(1.5) 


-Program Objective 


5 


-Program Objective 


5 


-Military Reqm'ts 


8 


-Military Reqm'ts 


8 


-MOU/Program 


10 


-MOU/Program 


10 


Schedules 




Schedules 




-Workshare 


5 


-Workshare 


5 


-Payoff 


8 


- Payoff 


8 


-Strength Rating 


10 . 80 


-Strength Rating 


10 . 80 


Abilitv to Contribute (1.0) 


Ability to Contribute 


(1.0) 


- Funding 


5 


- Funding 


9 


-Government 


8 


-Government 


8 


Technology 




Technology 




-Strength Rating 


6.50 


-Strength Rating 


8.50 


Technoloqv/ Security 


(1.0) 


Technoloqv/Securitv 


(1.0) 


-Technology 


8 


-Technology 


8 


Transfer 




Transfer 




-Security 


10 


-Security 


0 


-Strength Rating 


9 . 00 


-Strength Rating 


8.00 



OVERALL STRENGTH MATCHING 1.00 

COMPOSITE STRENGTH 62.10 



81 



PROGRAM SHORT TITLE: CSNI 



UNITED STATES 



Level of Commitment (0.75) 



-Congress 


3 


-OSD 


4 


-Service 


6 


-Strength Rating 


3.25 


Programmatic Factors 


(1.5) 


-Program Objective 


1 


-Military Reqm'ts 


7 


-MOU/ Program 


8 


Schedules 




-Workshare 


5 


- Payoff 


5 


-Strength Rating 


7.80 



Ability to Contribute (1.0) 



- Funding 


5 


-Government 


10 


Technology 




-Strength Rating 


7 . 50 



Technology /Security (1.0) 



-Technology 


10 


Transfer 




-Security 


5 


-Strength Rating 


7.5C 



FOREIGN 



Level of Commitment (0.75) 



-Legislature 


2 


-MOD 


5 


-Service/Equiv. 


6 


-Strength Rating 


3.25 



Programmatic Factors (1.5) 



-Program Objective 3 



-Military Reqm'ts 8 
-MOU/ Program 8 

Schedules 

-Workshare 5 

-Payoff 5 

-Strength Rating 8.70 

Ability to Contribute (1.0) 

-Funding 10 

-Government 10 

Technology 

-Strength Rating 10.00 

Technology/Security (1.0) 

-Technology 10 

Transfer 

-Security 0 

-Strength Rating 10.00 



OVERALL STRENGTH MATCHING 1.25 

COMPOSITE STRENGTH 62.38 
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PROGRAM SHORT TITLE: C2P2 



UNITED STATES 




FOREIGN 




Level of Commitment 


(0.75) 


Level of Commitment 


(0.75) 


-Congress 


1 


-Legislature 


1 


-OSD 


2 


-MOD 


4 


- Service 


4 


-Service/Equ iv. 


6 


-Strength Rating 


1.75 


-Strength Rating 


2.75 


Proqrammatic Factors 


(1.5) 


Proqrammatic Factors 


(1.5) 


-Program Objective 


1 


-Program Objective 


1 


-Military Reqm' ts 


1 


-Military Reqm'ts 


1 


-MOU/Program 


6 


-MOU/Program 


6 


Schedules 




Schedules 




-Workshare 


10 


-Workshare 


10 


- Payoff 


6 


- Payoff 


6 


-Strength Rating 


7.20 


-Strength Rating 


7.20 


Abilitv to Contribute (1.0) 


Abilitv to Contribute 


(1.0) 


-Funding 


2 


- Funding 


8 


-Government 


10 


-Government 


10 


Technology 




Technology 




-Strength Rating 


6 . 00 


-Strength Rating 


9.00 


Technoloqv/Securitv 


(1.0) 


Technoloqy /Security 


(i.Q) 


-Technology 


8 


-Technology 


8 


Transfer 




Transfer 




- Security 


5 


-Security 


0 


-Strength Rating 


6.50 


-Strength Rating 


8 . 00 



OVERALL STRENGTH MATCHING 1.25 

COMPOSITE STRENGTH 52.15 



83 



d. Results of Application of Revised Criteria and 

Method 

The results of the evaluation conducted using the revised 
criteria and method are summarized in the table below, 
compared, when applicable, to the results of the initial 
evaluation: 



PROGRAM 


INITIAL 


INITIAL 


REVISED 


REVISED 




EVALUATION 


RANK 


EVALUATION 


RANK 


NMMS 


59.51 


1 


75.20 


1 


SPIN 


53.00 


3 


68.50 


2 


ASMCM 


54.91 


2 


65.95 


3 


MUAV 


NA 


NA 


62.73 


4 


CSNI 


48.48 


5 


62.38 


5 


CIP 


50.50 


4 


62 . 10 


6 


ADV ST 


NA 


NA 


58.00 


7 


AL BAT 


NA 


NA 


55.38 


8 


C2P2 


35.94 


6 


52.15 


9 


RAIDS 


NA 


NA 


39.25 


10 



D. ANOMALIES RESULTING FROM APPLICATION OF REVISED CRITERIA 
AND METHOD 

Several anomalies are readily apparent regarding the two 
evaluation procedures. Of primary significance is the fact 
that the four programs being evaluated for the first time 
generally scored lower than those programs being re-evaluated 
using the revised procedures. Additionally, for two pairs 
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(ASMCM/SPIN and CIP/CSNI) , the revised rankings are the 
reverse of the initial rankings. These anomalies caused the 
IP office to question whether there might be problems with the 
revised procedure; evaluations using the revised procedure 
were halted until a review could be conducted. 
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IV: ANALYSIS OF DATA 



A. SPREADSHEET ANALYSIS OF INITIAL EVALUATION CRITERIA AND 

METHOD 

1. Construction of Spreadsheet 

To analyze the IPO evaluation process required translating 
the data from the various sets of evaluation sheets into a 
spreadsheet program that replicated both the appearance and 
the functions of the evaluation sheet. The cell contents of 
an example of this spreadsheet program are given below: 



Al: NATO MECHANICAL MINESWEEPING SYSTEM 

E3 : FOREIGN: FRANCE, SPAIN, 

A4 : UNITED STATES 

E4 : THE NETHERLANDS, ITALY 

A6 : LEVEL OF COMMITMENT 

E6 : LEVEL OF COMMITMENT 

A8 : CONGRESS 

C8 : 1 

E8 : LEGISLATURE 

G8 : 1 

A9 : OSD 

C9 : 3 

E9 : MOD 

G9 : 6 

A10: SERVICE 

CIO: 8 

E10: SERVICE/EQUIV. 

G10 : 9 

All: WEIGHT 

Bll : 0.75 

Ell: WEIGHT 

Fll : 0.75 

A12 : STRENGTH RATING 

C12 : @SUM(C8. .CIO) /@COUNT(C8. .CIO) *B11 

E12 : STRENGTH RATING 

G12 : ©SUM (G8 . . G10 ) /@COUNT (G8 . .G10) *F11 

A14 : PROGRAMMATIC FACTORS 
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E14 : PROGRAMMATIC FACTORS 

A16 : PROGRAM OBJECTIVES 

C16 : 4 

E16 : PROGRAM OBJECTIVES 

G16 : 4 

A17 : MILITARY RQMTS 

C17 : 10 

E17 : MILITARY RQMTS 

G17 : 10 

A18: ALLIANCE RQMTS 

C18 : 10 

E18 : ALLIANCE RQMTS 

G18 : 10 

A19 : MOU/ PROGRAM SKED 

C19 : 5 

E19 : MOU/ PROGRAM SKED 

G19 : 5 

A20 : INDUST BASE IMPACT 

C2 0 : 5 

E20 : INDUST BASE IMPACT 

G2 0 : 5 

A21: THIRD PARTY SALES 

C21 : 5 

E21 : THIRD PARTY SALES 

G21 : 5 

A22 : WEIGHT 

B22 : 1.5 

E22 : WEIGHT 

F22 : 1.5 

A23 : STRENGTH RATING 

C23 : @SUM(C16 . .C21) /@COUNT(C16 . .C21) *B22 

E23 : STRENGTH RATING 

G23 : @SUM (G16 . .G21) /@COUNT(G16 . .G21) *F22 

A25 : ABILITY TO CONTRIBUTE 

E25: ABILITY TO CONTRIBUTE 

A27: FUNDING 

C27 : 10 

E27: FUNDING 

G27: 10 

A28: GOVT TECHNOLOGY 

C28: 10 

E28: GOVT TECHNOLOGY 

G2 8 : 10 

A29 : INDUST TECHNOLOGY 

C29 : 8 

E29 : INDUST TECHNOLOGY 

G29 : 8 

A3 0 : TECH TRANSFER 

C30: 10 

E30 : TECH TRANSFER 

G30 : 10 
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A3 1 : SECURITY 

C31 : 

E31: SECURITY 

G31 : 5 

A3 2: WEIGHT 

B32 : 1 

E32 : WEIGHT 

F32 : 1 

A3 3 : STRENGTH RATING 

C33 : @SUM (C27 . .C31) /@C0UNT(C27. .C31) *B32 

E33 : STRENGTH RATING 

G33 : @SUM (G27 . .G31) /@C0UNT(G27. .G31) *F32 

A3 5: TOTAL STRENGTH 

C35 : +C12+C23+C33 

E35 : TOTAL STRENGTH 

G35 : +G12+G23+G33 

D38 : DEGREE OF CONVERGENCE 

D40 : COMBINED 

H40 : COMPOSITE 

D41 : STRENGTH 

F41: WEIGHT 

H41: WEIGHT 

A43 : LEVEL OF COMMITMENT 

D43 : @VALUE (C12) +@VALUE (G12) 

F43 : 1 

H4 3 : +D43 *F43 

A44 : PROGRAMMATIC FACTORS 

D44 : @ VALUE (C23) +@VALUE (G23) 

F44 : 1.25 

H44 : +D44*F44 

A45 : ABILITY TO CONTRIBUTE 

D45 : @VALUE (C33) +@VALUE (G33) 

F45 : 1.5 

H45 : +D45*F45 

E47 : TOTAL COMPOSITE STRENGTH 

H47 : +H43+H44+H45 



While creating the spreadsheet, it was discovered that, 
for any line item on the evaluation sheet not given a 
numerical score (i.e. "Security" under the United States' 
"Ability to Contribute"), the corresponding cell on the 
spreadsheet (C31) had to be completely blank. Any entry (i. 
e. NA, 0, or even a global justification command) in such a 
cell would be counted as an entry in the subsequent "@C0UNT" 
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formula used to compute the area strength rating (in this 
instance cell C33) and would result in an incorrect 
calculation. 

2. Results of Spreadsheet Analysis 

The key formula cells of this spreadsheet (C12, G12, C23, 
G23 , C33 , G33, C35, G35, and H47) duplicated the values of the 
area strength ratings, the total strength ratings, and the 
composite strength rating of the evaluation sheet. The values 
represented by the other formula cells (D43, H43, D44, H44, 

D45, H45) do not appear on the evaluation sheet but were 

included in the spreadsheet for ease of computation. 

This spreadsheet analysis was applied to the evaluation 
sheets representing the initial evaluation performed on the 
following candidate programs: 

• NATO Mechanical Minesweeping System 

• Anti-Ship Missile Countermeasure 

• "Spinnaker" 

• U. S. PACOM Combined Interoperability Program 

• NATO CSNI 

• Continuous Processing of Solid Propellants 

• Remote Sensing Unit with Synthetic Aperture Radar 

• Airborne Electromagnetic Sensor 

• Tactical Lightweight Surveillance Satellite 

• Tactical Target Support 

• Distributed Tactical Surveillance 
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• Tactical Ship Surveillance 

• Oceanography Cooperation- -Japan 

• Oceanography Cooperation- -Norway 

• Ocean Color Assessment (G-I-N Sea) 

• Ocean Color Assessment (Mediterranean Sea) 

• Millimeter Wave Sea Surveillance 

• Remote Spectrometry Research. 

In every instance, the spreadsheet analysis of these programs 
directly correlated to all values on the evaluation sheets. 

B. SPREADSHEET ANALYSIS OF REVISED EVALUATION CRITERIA AND 
METHOD 

1. Construction of Spreadsheet 

A second, similar spreadsheet based on the evaluation 
sheets of the revised evaluation criteria and procedures was 
then created; the criteria scores and weights of the Maritime 
Unmanned Aerial Vehicle (MUAV) evaluation sheet were used in 
creating the spreadsheet. The following represents the cell 
contents of this second spreadsheet: 

A1 : MUAV 

E2 : FOREIGN: CANADA, FRANCE 

E3 : GERMANY, NETHERLANDS 

A4 : UNITED STATES 

E4 : NORWAY, UNITED KINGDOM 

A6 : LEVEL OF COMMITMENT 

E6 : LEVEL OF COMMITMENT 

A8 : CONGRESS 

C8 : 1 

E8 : LEGISLATURE 
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G8 : 

A9 : OSD 

C9 : 1 

E9 : MOD 

G9 : 1 

A10: SERVICE 

CIO: 6 

E10: SERVICE/EQUIV. 

G10 : 6 

All: WEIGHT 

Bll : 0.75 

Ell: WEIGHT 

Fll : 0.75 

A12 : STRENGTH RATING 

C12 : @SUM(C8. .CIO) /@COUNT(C8. .CIO) *B11 

E12 : STRENGTH RATING 

G12 : @SUM(G8 . .G10) /@COUNT (G8 . . G10 ) * Fll 

A14 : PROGRAMMATIC FACTORS 

E14 : PROGRAMMATIC FACTORS 

A16 : PROGRAM OBJECTIVES 

C16 : 8 

E16 : PROGRAM OBJECTIVES 

G16: 8 

A17 : MILITARY RQMTS 

C17 : 8 

E17 : MILITARY RQMTS 

G17 : 7 

A18 : MOU/PROGRAM SKED 

Cl 8 : 10 

E18 : MOU/PROGRAM SKED 

G18 : 10 

A19 : WORKSHARE 

C19 : 5 

E19 : WORKSHARE 

G19 : 5 

A20 : PAYOFF 

C2 0 : 8 

E20: PAYOFF 

G2 0 : 8 

A21 : WEIGHT 

B21: 1.5 

E21: WEIGHT 

F21: 1.5 

A22 : STRENGTH RATING 

C22 : @SUM ( C16 . .C20) /@COUNT(C16. .C20) *B21 

E22 : STRENGTH RATING 

G22 : @SUM (G16 . .G20) /@COUNT(G16. .G20) *F21 

A24 : ABILITY TO CONTRIBUTE 

E24 : ABILITY TO CONTRIBUTE 

A26 : FUNDING 

C26: 10 
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E26: FUNDING 

G2 6: 5 

A27: GOVT TECHNOLOGY 

C27 : 

E27: GOVT TECHNOLOGY 

G27 : 

A28: WEIGHT 

B28 : 1 

E28: WEIGHT 

F28 : 1 

A2 9 : STRENGTH RATING 

C29: 0SUM(C26. . C27) / @ COUNT (C26. . C27) *B28 

E2 9: STRENGTH RATING 

G2 9: 0SUM (G2 6 . . G27) / 0COUNT (G2 6. .G27) *F28 

A3 1 : TECHNOLOGY/SECURITY 

E31: TECHNOLOGY/SECURITY 

A33: TECH TRANSFER 

C33 : 10 

E33: TECH TRANSFER 

G33: 10 

A3 4: SECURITY 

C34 : 10 

E34: SECURITY 

G34 : 

A35: WEIGHT 

B35: 1 

E35: WEIGHT 

F35: 1 

A3 6 : STRENGTH FATING 

C3 6 : 0SUH (C33 . .C34) /0COUNT (C33. . C34) *B35 

E36: STRENGTH FATING 

G36: 0SUM (G33 . -G34) /0COUNT(G33. .G34) *F35 

A40: TOTAL COMBINED STRENGTH 

D40 : +C12+G12+C22+G22+C29+G2 9+C3 6+G3 6 

A41: STRENGTH MATCHING 

D41: 1 

A* 3: TOTAL COMPOSITE STRENGTH 

Dd 3 : +D40-D41 



2 . Results of Spreadsheet Analysis 

With this spreadsheet, the formula cells corresponding to 
the area strength ratings (C12, G12, C22, G22, C29, G29, C36 
and G36) produced values identical with the values on the 
evaluation sheet. The composite strength formula cell (D43) , 
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however, produced a value different from the evaluation sheet 
(97.80 vice 75.20). This second spreadsheet was applied to 
all the programs that had undergone the revised evaluation 
procedure, using the criteria scores and weights from the 
evaluation sheets. Always, the results were the same: area 
strength ratings corresponded to the evaluation sheets but 
composite strength ratings differed. The following table 
summarizes the results of this step in the analysis: 



PROGRAM 

SHORT 

TITLE 


SPREAD- 

SHEET 

COMPOSITE 

STRENGTH 

RATING 


SPREAD- 

SHEET 

RANKING 


EVALUA- 

TION 

SHEET 

COMPOSITE 

STRENGTH 


EVALUA- 

TION 

SHEET 

RANKING 








BATING 




NMMS 


97 . 80 


1 


75.20 


1 


ASMCM 


86.18 


2 


65.95 


3 


SPIN 


80.00 


3 


68.50 


2 


CSNI 


72.50 


4 


62.38 


5 


ADV ST 


67.66 


5 


58.00 


7 


MUAV 


62.73 


6 


62.73 


4 


CIF 


62.10 


3 


62.10 


6 


C2P2 


60.50 


8 


52.15 


9 


AL BAT 


55.38 


9 


55 .38 


8 


PAIRS 


39.25 


10 


39.25 


10 
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Since the spreadsheet calculations corresponded to the 
evaluation sheets in all area strength ratings, and differed 
in all composite strength ratings, the cell governing the 
computation of "Composite Strength" was examined further. 

C. REASON FOR DISCREPANCY BETWEEN EVALUATION SHEETS AND 
SPREADSHEET ANALYSIS 

The "Composite Strength" value on the evaluation sheet was 
being calculated in a manner different from the intent of the 
evaluation sheet. Referring back to the MUAV spreadsheet 
analysis, "Composite Strength" is calculated by the formula 
"+D40*D41" or, substituting in for cell D40, by the formula 

"+ ( C12 +G12+C22 +G22 + C29 +G29 + C3 6+G3 6 ) *D41. " 

This formula captures the intent of the revised procedure: to 
apply a single "Strength Matching" weight (D41) to the sum of 
all the area strength ratings. 

The IPO evaluation sheets, however, calculate "Composite 
Strength" using the formula 

"+C12+G12+C22+G22+C36+G36+ ( (C29 +G29 ) *D4 1 ) . " 

This formula isolates and sums the "Ability to Contribute" 
area strength ratings, applies the "Strength Matching" weight 
to that sum, and then adds in the remaining area strength 
ratings. By doing so, the formula is responsible for the 
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downward skewing noticed by the IP Office when they first 
applied the revised evaluation criteria. 

D. APPLYING THE SPREADSHEET CALCULATIONS TO THOSE CANDIDATE 

PROGRAMS NEVER EVALUATED WITH REVISED CRITERIA AND METHOD 

The spreadsheet program representing the intent of the 
revised criteria and procedure was then applied to the twelve 
programs never re-evaluated with the revised criteria. 
Criteria scores for the two line items ("Workshare" and 
"Payoff") not on the initial evaluation sheet had to be 
developed. For "Workshare, " a value equivalent to the score 
given to "Industrial Base Impact" on the initial evaluation 
was used. For "Payoff," the subjectively determined "Payoff 
Potential" from the initial evaluation was translated into a 
numerical score between one and ten and used in the 
spreadsheet. The "Degree of Convergence" weight assigned to 
the "Ability to Contribute" area on the initial evaluation was 
used as the "Strength Matching" weight of the spreadsheet. 
The "Workshare" and "Strength Matching" assignments were done 
after consulting with Mr. Frank Kenlon of the IP Office; the 
"Payoff" assignment represents a natural progression from one 
format to another. The results of this analysis are shown in 
the table below: 
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PROGRAM 


STRENGTH 


COMPOSITE 


SHORT TITLE 


WEIGHTING 


STRENGTH 

RATING 


RSU 


1.25 


76.25 


AES 


1.25 


69.06 


TLSS 


1.25 


58 . 63 


DTS 


1.00 


53.90 


TTS 


1.25 


53.25 


OCA-G 


1.00 


50.35 


OCA-M 


1.00 


50.35 


OC-J 


1.00 


48.50 


OC-N 


1.25 


48.50 


TSS 


0.75 


41.03 


MWSS 


1.00 


39.20 


RSR 


1.00 


38 . 70 



E. FINAL PRIORITY RANKING OF ALL 22 CANDIDATE PROGRAMS 

The following table represents the final ranking of the 22 
programs based on the composite strength ratings calculated by 
the spreadsheet; included is a column with the current status 
of each candidate program: 



PROGRAM 
SHORT TITLE 


COMPOSITE 

STRENGTH 

RATING 


CURRENT 

STATUS 


NMMS 


97.80 


SIGNED MOU 


ASMCM 


86.18 


AWAITING START 
OF NEGOTIATIONS 


SPIN 


80.00 


IN NEGOTIATION 


RSU 


76.25 


PROPOSAL 

WITHDRAWN 
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PROGRAM 
SHORT TITLE 


COMPOSITE 

STRENGTH 

RATING 


CURRENT 

STATUS 


CSNI 


72.50 


SIGNED MOU 


AES 


69.06 


j 

PROPOSAL 

WITHDRAWN 


ADV ST 


67.66 


DEVELOPING RAN 


MUAV 


62.73 


DEVELOPING RAN 


CIP 


62.10 


AWAITING START 
OF NEGOTIATIONS 


C2P2 


60.50 


DEVELOPING RAN 


TLSS 


58.63 


PROPOSAL 

WITHDRAWN 


AL BAT 


55.38 


DEVELOPING RAN 


DTS 


53.90 


PROPOSAL 

WITHDRAWN 


TTS 


53.25 


PROPOSAL 

WITHDRAWN 


OCA-G 


50.35 


PROPOSAL 

WITHDRAWN 


OCA-M 


50.35 


PROPOSAL 

WITHDRAWN 


OC-J 


48.50 


PROPOSAL 

WITHDRAWN 


OC-N 


48.50 


PROPOSAL 

WITHDRAWN 


TSS 


41.03 


PROPOSAL 

WITHDRAWN 


RAIDS 


39.25 


PROPOSAL 

WITHDRAWN 


MWSS 


39.20 


PROPOSAL 

WITHDRAWN 


RSR 


38.70 


PROPOSAL 

WITHDRAWN 
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With the spreadsheet evaluation, it appears that a 
composite strength score of 55 or above represents a 
threshold; all programs that did not reach that threshold 
resulted in the candidate program being withdrawn from 
consideration. Two programs (RSU and AES) that exceeded the 
threshold also had their proposals withdrawn; to correlate 
their withdrawal to their evaluation scores, statistical 
analysis techniques were applied to the spreadsheet evaluation 
scores . 

F. STATISTICAL ANALYSIS OF SPREADSHEET EVALUATION SCORES 

Based on the status of each of the 22 programs, the 
programs were assigned to one of two groups, Unsuccessful or 
Successful, to perform a statistical analysis of the 
spreadsheet evaluation scores. The Unsuccessful group 
consisted of those programs with a status indicating that the 
proposal had been withdrawn; the Successful group consisted of 
all others. 

Combined (U. S. and Foreign) median and sample mean scores 
for each of the four evaluation areas of the spreadsheet 
(Level of Commitment, Programmatic Factors, Ability to 
Contribute, and Technology/Security) were calculated for each 
group; the standard distribution of the mean was also 
calculated. The following four tables record the results of 
those calculations: 
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COMBINED LEVEL 


UNSUCCESSFUL 


SUCCESSFUL 


OF COMMITMENT 


PROPOSALS 


PROPOSALS 


MEDIAN 


3.50 


4.63 


SAMPLE MEAN 


3 . 08 


5 . 54 


STANDARD 


0.88 


2 . 00 


DEVIATION 







j COMBINED 
j PROGRAMMATIC 
! FACTORS 


UNSUCCESSFUL 

PROPOSALS 


SUCCESSFUL 

PROPOSALS 


MEDIAN 


15 . 00 


20.70 


SAMPLE MEAN 


15 . 44 


19.50 


STANDARD 

DEVIATION 


2.47 


3 .87 



COMBINED 


UNSUCCESSFUL 


SUCCESSFUL 


ABILITY TO 


PROPOSALS 


PROPOSALS 


CONTRIBUTE 






MEDIAN 


13.00 


17 . 00 


SAMPLE MEAN 


13 . 12 


16 .67 


STANDARD 


2.36 


1.71 


DEVIATION 







COMBINED 


UNSUCCESSFUL 


SUCCESSFUL 


TECHNOLOGY/ 


PROPOSALS 


PROPOSALS 


SECURITY 






MEDIAN 


20.00 


17.50 


SAMPLE MEAN 


16 . 73 


17 . 00 


STANDARD 


4.38 


2.49 


DEVIATION 







Of all the median and sample means calculated, the most 
statistically significant, in terms of the range of score 
disparity between the Unsuccessful and Successful groups, are 
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the sample mean scores of the Level of Commitment area. 
Combined median, sample means, and standard distributions were 
calculated for each line item in the Level of Commitment area; 
the results are shown in the three tables below: 



COMBINED 


UNSUCCESSFUL 


SUCCESSFUL 


CONGRESS/ 


PROPOSALS 


PROPOSALS 


LEGISLATURE 






MEDIAN 


2 . 00 


2.00 


SAMPLE MEAN 


2 . 00 


2.22 


STANDARD 


0.00 


1.09 


DEVIATION 







COMBINED 


UNSUCCESSFUL 


SUCCESSFUL 


OSD/MOD 


PROPOSALS 


PROPOSALS 


MEDIAN 


2 . 00 


8.00 


SAMPLE MEAN 


2 . 00 


7 . 33 


STANDARD 


0 . 00 


4 . 09 


DEVIATION 







COMBINED 


UNSUCCESSFUL 


SUCCESSFUL 


SERVICE/ 


PROPOSALS 


PROPOSALS 


EQUIVALENT 






MEDIAN 


10.00 


12.00 


SAMPLE MEAN 


8.46 


11.44 


STANDARD 


3.38 


4.69 


DEVIATION 







Statistically, the most significant of these line item 
scores are the median and sample mean values of the combined 
OSD/MOD line item scores. Without OSD/MOD support, a proposal 
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appears doomed; with OSD/MOD support, a proposal's weaknesses 
in other areas may be surmounted. 



G. CORRELATION OF IPO'S EVALUATION CRITERIA WITH OSD SUCCESS 
INDICATORS 

At a Four Powers (United States, United Kingdom, France, 
Germany) Conference held in Monterey, California on October 5, 
1991, the Deputy Under Secretary of Defense for International 
Programs (DUSD(IP)) briefed the participants on OSD's 
viewpoint regarding the criteria of success for cooperative 
programs; the OSD criteria are presented below: 

• A common, well defined requirement; 

• An early start supported by technology cooperation and 
venture capital from Nunn amendment funding; 

• A clear view ahead on life-cycle funding; 

• A clear view of competitive programs; 

• Equal priority for each player; 

• Two or three partners maximum; 

• Each partner bringing something to the program besides 
money ; 

• The will to see it through, including a mutual 
understanding of cultures, concurrence on acquisition 
strategy, strong program managers, and support at the 
highest level. 



The policy, procedures and criteria implemented by IPO 
through the evaluation of candidate programs generally 
supports the OSD success criteria. One instance of divergence 
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pertains to the number of partners being considered for a 
program; IPO does not take into account a candidate program 
having more than three partners while conducting its 
evaluation. Unless it has significant OSD/MOD support, a 
program with an excessive number of partners may experience 
difficulties obtaining the authority to negotiate an 
international agreement. 
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V. 



CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

The examination of the evaluation performed by the Navy 
International Programs Office on 22 candidate program 
proposals led to the following conclusions: 

• Upon revising the success criteria, a miscommunication 
resulted in evaluations being conducted in such a manner 
that overemphasized the "Ability to Contribute" area of 
the criteria and that produced inaccurate "Composite 
Strength" scores and ranking. 

• The single, evaluation criteria line item that principally 
determines whether a candidate program gains approval to 
develop the Request Authority to Negotiate document is the 
level of OSD/MOD support of the program; by extension, the 
single principal evaluation area is the "Level of 
Commi tment " area . 

• The International Programs Office's policy, procedures, 
and evaluation criteria are in general agreement with 
their OSD counterparts; however, the International 
Programs Office does not penalize a candidate program for 
having an excessive number of partners. 



B . RECOMMENDATIONS 

The recommendations derived from the above conclusions are 
as follows: 

• The International Programs Office should correct the 
software programming error that produces the evaluation 
anomalies; this should be accomplished by implementing the 
spreadsheet "Composite Strength" formula in Section IV-C 
of this thesis. 

• The International Programs Office should direct its 
resources at those candidate programs with a composite 
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strength score of 55 or above, and at all candidate 
programs with a combined OSD/MOD line item score of three 
or above . 

• The International Programs Office should subtract points 
from the composite strength scores of those candidate 
programs with an excessive number of partners before 
ranking to reflect the greater risk of the program. 

• The International Programs Office should continue to 
critique the criteria and procedures used to evaluate 
candidate programs to stay current with the rapidly 
changing environment of the international arena. 
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